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I I 


FOREWORD 


The  Battlefield  Information  Systems  Technical  Area  is  concerned 
with  the  demands  of  the  future  battlefield  for  increased  man-machine 
complexity  to  acquire,  transmit,  process,  disseminate,  and  use  informa- 
tion. The  research  focuses  on  the  interface  problems  and  interactions 
within  command  and  control  centers  and  deals  with  such  areas  as  topo- 
qraphic  products  and  procedures,  tactical  symboloqy,  information  manaqe- 
ment,  user-oriented  systems,  staff  operations  and  procedures,  and  sensor 
systems  inteqration  and  utilization. 

One  area  of  special  interest  is  that  of  human  factors  problems  of 
the  presentation  and  interpretation  of  surveillance  and  tarqet  acquisi- 
tion information.  One  relatively  new  source  of  intelliqence  information 
is  remote  monitorinq  of  the  battlefield  usinq  seismic,  acoustic,  and 
maqnetic  unattended  qround  sensors  (UGS) . When  these  remote  sensors 
are  activated  by  enemy  personnel  or  vehicle  movement,  a monitor  display 
located  behind  our  lines  indicates  the  activity.  The  operator  can  de- 
rive from  this  display  not  only  the  presence  of  the  enemy  but  such  in- 
formation as  the  direction  and  speed  of  convoys  and  personnel,  the 
number  of  vehicles  in  a convoy,  and  the  composition  of  the  convoy,  e.q., 
armored  versus  wheeled  vehicles.  The  present  publication  discusses  the 
development  and  evaluation  of  a traininq  test  and  job  aids  which  siq- 
nificantly  improve  operator  performance  by  correctinq  known  deficiencies. 
In  addition,  the  number  of  sensors  in  a strinq  and  level  of  tarqet 
activity  was  varied  in  the  research  to  determine  their  effects  on  oper- 
ator performance. 

Research  in  the  area  of  sensor  systems  inteqration  and  utilization 
is  conducted  as  an  in-house  effort  auqmented  throuqh  contracts  with  or- 
ganizations selected  for  their  unique  capabilities  and  facilities  for 
research  on  sensor  systems.  The  present  study  was  conducted  with  per- 
sonnel from  HRB  Sinqer,  Inc.  under  contract  DAHC  19-73-C-0024  with  pro- 
qram  direction  from  Mr.  Cecil  D.  Johnson.  The  effort  is  responsive  to 
requirements  of  Army  Project  2Q662704A721  and  to  special  requirements 
of  the  Assistant  Chief  of  Staff  for  Intelliqence. 

The  cooperation  of  the  participation  personnel  at  Fort  Braqq,  N.C., 
is  appreciated,  especially  that  of  Maj.  Gronich.  A special  note  of 
thanks  is  due  to  Sqt.  D.  Chiodini,  chief  of  the  Read-out  Section  of  the 
UGS  Division,  U.S.  Army  Intelliqence  Center  and  School  (USAICS) , Fort 
Huachuca,  Ariz.,  for  his  expert  technical  assistance. 
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BRIEF 


Requirement: 

To  determine  the  value  of  unattended  ground  sensor  operator  train- 
ing and  job  aids  derived  from  an  analysis  of  error  sources  and  the  ef- 
fect on  operator  performance  of  target  activity  level  and  numbers  of 
sensors  used  in  a string. 


Procedure : 

Based  on  an  analysis  of  operator  errors  made  in  a previous  study, 
a training  program  and  two  job  aids  were  developed.  To  test  the  value 
of  the  training  and  one  of  the  job  aids  (measuring  device) , two  2-hour 
scenarios  were  constructed  for  pre-  and  posttraining  evaluation.  Typi- 
cal target  patterns  at  two  levels  of  target  activity  and  three  levels 
of  sensor  string  size  were  systematically  varied  within  30-minute  seg- 
ments. Authentic  fixed-wing  and  helicopter  activity,  artillery  shell 
bursts,  and  random  noise  were  included  to  simulate  operational,  nontar- 
get activations.  The  second  job  aid,  a nomograph,  was  evaluated  usinq 
the  pretest  and  posttest  design.  Two  special  tests  requiring  only  the 
measurement  and  computations  necessary  for  estimates  of  speed  and  enemy 
number (s)  were  developed  for  this  purpose. 

Twenty  school- trained  Army  enlisted  men  (UGS  (unattended  ground 
sensor)  operators) , were  given  test  procedure  training  and  a short  re- 
fresher in  UGS  interpretation.  Two  10-man  groups  were  formed  and  each 
was  given  different  scenarios  for  the  pretest;  these  were  then  switched 
for  the  posttest.  The  training  program — given  between  the  pretest  and 
posttest — stressed  individualized  instruction  includinq  self-pacinq, 
immediate  feedback,  expert  assistance  when  needed,  and  guaranteed 
student-mastery  using  criterion  testinq.  One  job  aid,  an  UGS  ruler 
for  accurately  measuring  the  length  of  activation  patterns,  was  part 
of  the  training  exercise.  The  other  job  aid  (a  nomoqraph) , which  had 
been  developed  to  simplify  arithmetic  calculations  and  decimal  point 
placement,  was  tested  separately  after  the  above  traininq  and  posttest. 


Findings: 

The  individualized  training  program  resulted  in  significantly  im- 
proved operator  interpretation  performance  in  target  detection  rights. 


identification  riqhts,  tarqet  speed,  and  tarqet  quantity  estimation. 

Use  of  the  nomoqraph  siqni f icantly  improved  performance  in  identifica- 
tion riqhts,  tarqet  speed  estimation,  tarqet  quantity  estimation,  and 
reportinq  time.  Student  acceptance  of  the  individualized  traininq  ap- 
proach and  of  both  job  aids  was  hiqh.  Operator  performance  on  3-  and 
4-sensor  strinqs  was  77%  detection  completeness  with  virtually  no  false 
alarms.  Detection  of  tarqets  was  better  durinq  low  tarqet  activity 
than  durinq  hiqh  tarqet  activity.  Use  of  three  sensors  in  a strinq 
resulted  in  the  same  operator  performance  as  use  of  four  sensors. 


Utilization  of  Findinqs: 

The  lesson  materials  together  with  the  individualized  trainino  ap- 
proach should  be  used  to  provide  review  and  on-the-job  traininq  to  oper- 
ational field  personnel  and  should  also  be  inteqrated  into  the  UGS 
course  at  USAICS,  Fort  Huachuca,  Ariz. 

The  nomoqraph  should  be  included  as  standard  issue  with  opera- 
tional event  recorders  and  should  be  tauqht  at  UGS  course  USAICS  and 
qiven  to  each  qraduate  to  take  with  him  to  his  assiqned  unit.  A credit- 
card  size  UGS  ruler  should  be  issued  to  everyone  involved  in  monitorinq 
UGS  tarqet  activation. 

To  increase  timeliness,  the  UGS  operator  should  send  forward  a 
detection  report  based  on  two  sensors  (or  more,  if  doubt  exists)  with 
a followup  report  giving  tarqet  type,  speed,  and  number. 

A research  study  controlling  on  tarqet  difficulty  and  comoarinq 
2 versus  3 versus  4-sensor  strinqs  should  be  done  to  provide  data  for 
determining  the  most  efficient  strinq  size. 

During  hiqh-target-activity  conditions,  operator  reports  should 
be  considered  as  qreater  underestimates  than  durinq  low-tarqet-acti vity 
periods . 
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THE  VALUE  OF  SPECIAL  TRAINING  AND  JOB  AIDS  FOR 
IMPROVING  UNATTENDED  GROUND  SENSOR 
OPERATOR  PERFORMANCE 


BACKGROUND 

As  discussed  in  the  ST  30-20-1  Army  manual  (U.S.  Army,  1971),  in 
September  1966  the  Secretary  of  Defense  directed  the  armed  forces  to 
develop  an  infiltration  interdiction  capability  to  assess  vehicle  and 
personnel  flow  in  Southeast  Asia.  The  anti-infiltration  capability  that 
was  developed  centered  on  a croup  of  remotely  monitored  electronic  sen- 
sor devices  called  unattended  ground  sensors  (UGS) . 

UGS  provides  the  Army's  battlefield  surveillance  and  tarqet  acquisi- 
tion system  with  another  versatile  capability,  that  of  obtaininq  real- 
time information  about  enemy  movement.  Most  UGS  in  use  today  are  seis- 
mic, others  are  maqnetic,  acoustic,  electromaqnetic , and  passive 
infrared.  They  can  be  hand  implanted,  air  implaced,  or  mortar  and 
artillery  delivered.  UGS  systems  are  recognized  as  effective  battle- 
field assets,  and  they  are  expected  to  become  an  important  source  of 
combat  intelliqence  for  many  situations. 

Even  thouqh  UGS  have  already  demonstrated  their  value  in  opera- 
tional and  test  situations  (U.S.  Army,  1971  and  1972),  their  informa- 
tion potential  is  far  greater  than  has  been  realized  heretofore.  Con- 
trolled tests  show  that  varyinq  deqrees  of  misleading  and  inaccurate 
information  are  being  supplied  by  both  the  sensor  equipment  and  human 
operators  (Martinek,  Pilette,  and  Biqqs,  1978;  U.S.  Army,  1970;  and 
West,  1973) . Appropriate  research  and  development  can  minimize  such 
deficiencies . 

Systematic  field  experimentation  shows  that  UGS  can  bo  unreliable 
at  times.  One  reason  is  lack  of  understanding  and  anticipation  of  en- 
vironmental influences  upon  sensor  response  characteristics.  For  exam- 
ple, in  a field  test  conducted  under  dry-season  terrain  conditions, 
considerable  site-to-site  variation  occurred  in  seismic  response  char- 
acteristics due  to  soil  surface  conditions,  veqetation  ground  covers, 
and  background  noises  (West,  1973) . 

Variable  sensor  response  characteristics  make  the  job  more  diffi- 
cult for  the  sensor  operator,  and  human  error  has  definitely  contributed 
to  inaccurate  and  misleading  information  output.  The  problem  is  to 
isolate  techniques  that  will  reduce  human  error  in  UGS  tarqet  reporting. 
Althouqh  a number  of  diverse  factors  may  make  UGS  operator  performance 
less  than  optimal,  the  more  prominent  problems  can  be  attacked  directly 
through  specialized  training,  operator  techniques,  and  job  aids. 


1 


OBJECTIVES 


1,  To  Identify  problem  areas  experienced  by  DOS  operators,  de- 
velop specialized  traininq  materials  to  control  these  diffi- 
culties, and  empirically  demonstrate  the  effectiveness  of 
such  traininq. 

2.  To  determine  if  two  lob  aids  (nomograph  and  UvX?  ruler)  can 
Improve  the  measurements  and  computations  of  DOS  monitors. 

.1.  To  decrease  the  time  needed  for  UGS  reporting. 

4.  To  determine  parameters  of  performance  under  several  opera- 
t.  ions  1 condi  t ions . 

5.  To  determine  the  changes  in  performance  due  to  the  number  of 
sensors  used  in  a string. 


METHOD 

Population  and  Sample 

The  population  of  concern  is  the  UGS  operator  ( MOS  17M20)  who  has 
been  school  trained  at  the  U.S.  Army  Intelligence  Center  and  School 
(USAICS) , Port  Huachuca,  Ariz.  TVenty  enlisted  personnel  MOS  17M20 
of  the  R2nd  Airborne  Division  stationed  at  Fort  Bragg,  N.C.,  served 
as  operators.  Although  they  had  been  in  the  field  for  several  months, 
they  had  not  interpreted  tins  activations  vising  the  operational  dis- 
play (th«r  R0376  event  recorder). 


Apparatus 

Ten  modified  Esterline  Angus  Chart  Inspectors  were  used  to  display 
the  pretest  and  posttest  scenarios  (R0376  event  recorder  plots) . Each 
was  mechanically  adjusted  to  drive  the  plots  at  the  same  speed  (12 
Inches  per  hour)  as  the  R037R  event  recorder  (U.S.  Army,  l^no) . A 
display  housing  was  placed  on  each  mechanism  to  reduce  its  viewing 
area  to  that  of  the  R0376.  Thus,  the  test  scenarios  were  presented 
at  the  same  speed  aivd  format  as  in  a field  situation. 


Train! nc 


Review,  live  first  step  in  the  identification  of  problem  areas  in 
UvJS  monitor  performance  involved  a review  of  training  being  conducted 
at  Fort  Huachuca.  Training  manual  documentation  was  reviewed  (U.S. 

Army,  1<»71,  l*)72)  and  interviews  were  held  with  senior  instructors  to 
identify  specific  areas  of  substandard  i^er formance . Student  performance 
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data,  collected  during  an  earlier  experiment  (Marti nek,  Pilette,  and 
Biqqs,  1978),  were  analyzed  in  an  effort  to  document  and  categorize 
factors  leadinq  to  human  error  in  UGS  monitor  performance.  Based  on 
these  efforts,  three  malor  problem  areas  were  identified. 

Target  Detection  and  Tdent  i f icat  ion . Tills  category’  deals  wi  th  the 
operator's  ability  to  detect  and  identify  targets.  The  operators  de- 
tected fewer  targets  when  these  targets  activated  2-sensor  strings 
as  compared  to  3-  and  4-sensor  strings.  Also,  lower  detection  com- 
pleteness (detection  rights  divided  by  total  number  of  targets)  oc- 
curred during  high  target  activity  as  compared  to  low  target  activity. 
Also,  approximately  20%  of  the  targets  detected  were  not  identified 
correctly  as  to  vehicle  or  personnel.  Following  the  pattern  established 
by  the  detection  results,  fewer  tarqets  were  identified  when  they  were 
activated  with  2-sensor  strings  and  during  high  target  activity. 

Column- Length  Formula  Calculations  (see  Appendix  A)  . This  cate- 
gory deals  with  the  ability  of  the  operator  to  use  the  column-length 
formula  with  target  patterns  that  he  has  detected.  A sensor  activation 
pattern  is  composed  of  short  horizontal  lines  occurring  within  a speci- 
fied time  interval  (determined  by  a sensor's  inhibit  time) . They  are 
displayed  on  the  appropriate  channel  of  the  plot  paper  which  serves  as 
the  readout  sheet  for  the  operator.  The  formula  provides  an  estimated 
tarqet  speed  and  number  of  targets  in  a convoy.  Some  specific  factors 
noted  were  (1)  incorrect  placement  of  the  endpoints  of  sensor  activa- 
tion patterns  for  measuring  TTj  and  TM  (see  page  10  for  definition) > 

(2)  inaccurate  estimations  of  TTi  and  TM  measurements,  even  if  the 
activation  end-points  are  not  ambiquous;  (3)  arithmetic  errors  in 
multiplication,  division,  and  decimal  point  placement;  and  (4)  con- 
fusion in  computing  the  combined  detection  range  (CDR)  value  and  in 
selecting  the  arithmetic  function  to  use  with  it. 

Nonseismic  and  Confirmatory  Sensors.  This  category  deals  with  the 
inability  of  the  operator  to  optimally  use  information  presented  by’  the 
following  sensors  when  these  sensors  were  part  of  a deployed  sensor 
strinq:  electromagnetic  intrusion  detector  (EMIP) , magnetic  intrusion 
detector  (MAGID) , and  directional  infrared  intrusion  detector  (DIRTD) . 


Development  of  Training  Materials  and  Job  Aids 

Althouqh  a number  of  training  requirements  could  be  satisfied  by 
the  development  of  content  in  specific  subject  matter  areas,  analysis 
of  traininq  requirements  revealed  a need  for  job  aids  that  would  assist 
the  UGS  monitor  in  makinq  measurements  and  calculations  relating  to  the 
column- length  formula. 
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The  following  content  lessons  were  outlined  and  developed. 

a.  Lesson  1 — UGS  Ruler  (traininq  in  the  use  of  this  job  aid)  . 
Objective — To  improve  accuracy  in  measurinq  TTj  and  TM  values 
by  usinq  a calibrated  ruler. 

b.  Lesson  2 — 2-Sensor  Strinqs  and  Formula  Review. 

Objective — To  improve  detection  capability  usinq  2-sensor 
strinqs  and  to  provide  recall  or  basic  traininq  in  comput- 
inq  the  column- lenqth  formula. 

c.  Lesson  3 — End-Points. 

Objective — To  standardize  and  improve  estimations  of  end- 
point locations  in  the  case  of  irreqular  activation  patterns. 

d.  Lesson  4 — Column- Lenqth  Quantity. 

Objective — To  review  and  to  provide  more  extensive  traininq 
in  interpretinq  variations  in  quantity  values. 

e.  lesson  5 — Irreqular  Activation  Patterns. 

Objective — To  improve  the  detection  and  information  extrac- 
tion from  tarqet  patterns  that  do  not  have  the  classical , 
clear-cut  shape  (stair  step) . 

f.  Lesson  6 — F.MTD. 

Objective — To  increase  information  extraction  potential  of 
the  F.MID  sensor  throuqh  familiarization  with  its  activa- 
tion patterns. 

q.  Lesson  7 — MAGID. 

Objective — To  illustrate  the  MAGID  capability  and  limita- 
tions in  providinq  information  concerninq  the  number  of 
tarqets  in  a column. 

h.  Lesson  8 — DIRID. 

Objective — To  increase  the  information  potential  of  the 
DIRID  sensor  throuqh  familiarization  with  its  activation 
fiat  terns. 

The  error  analysis  Indicated  a need  to  (a)  accurately  measure  the 
elapsed  time  of  tarqet  activation  patterns  that  are  formed  from  UGS  and 
(b)  reliably  perform  arithmetic  functions  and  decimal  point  placement. 
To  satisfy  the  former  requirement,  an  UGS  ruler  was  developed  that 
could  be  laid  down  on  the  R0376  plot  paper  to  permit  activation  pat- 
terns to  be  measured  consistently  and  accurately  to  the  nearest  1/10 
minute.  Althouqh  use  of  the  UGS  ruler  was  tauqht  in  Lesson  1,  it  was 
also  included  as  a part  of  the  nomoqraph  developed  to  satisfy  the  sec- 
ond need  above.  Thus,  the  operator  would  need  only  one  instrument  for 
measurement,  arithmetic  calculations , and  decimal  point  placement  as 
required  by  the  column- lenqth  formula. 


The  nomograph  is  shown  in  Figure  1.  The  abscissa  is  the  distance 
between  the  seismic  sensors  (in  meters) . The  ordinate  is  target  acti- 
vation time  (in  minutes) . A slide  indicator  or  tongue,  consisting  of 
a clear  plastic  rectangle  3/4  in.  x 10-1/2  in.  with  a black  line  running 
through  the  long  axis,  is  anchored  by  a grommet  in  the  upper  left-hand 
corner  from  where  it  pivots.  For  computing  target  speed,  — ^ of  the 
column- length  formula,  the  slide  indicator  is  pivoted  until  the  black 
line  is  over  the  intersection  of  the  activation  time  and  the  sensor 
distance.  The  target  speed  can  then  be  read  from  the  meters-per- 
minute  scale  at  the  bottom  and  right-hand  side  of  the  nomograph. 

With  the  slide  indicator  remaining  in  the  same  position,  the  nomo- 
graph can  then  be  used  to  compute  (TT1)  of  the  formula.  This  is  done 
simply  by  locating  the  intersection  of  the  black  line  on  the  slide  indi- 
cator with  the  TTj  time  measurement,  and  then  reading  the  distance  value 
from  the  meters  scale.  The  difference  between  this  value  and  the  CDR 
is  the  target  column- length . The  nomograph,  therefore,  can  be  used  for 
division  and  multiplication.  The  Fort  Huachuca  training  personnel  have 
developed  additional  uses  for  the  nomograph  in  various  applications. 


Design  of  the  Training  System 

With  the  training  content  determined,  the  next  step  was  to  design 
the  training  system  with  which  to  implement  the  content.  The  training 
system  was  designed  to  allow  each  individual  to  proceed  at  his/her  own 
pace,  test  knowledge  after  short  segments  (lessons)  of  instruction, 
obtain  immediate  feedback  on  mastery  of  the  content,  and  receive  ex- 
pert assistance  when  needed.  The  training  system  is  described  below 
in  terms  of  its  three  major  components:  materials,  personnel,  and 
lesson  sequence. 

In  the  self-paced  training  system  that  was  designed,  the  content 
was  organized  into  distinct  lessons,  each  covering  a short  segment  of 
instruction.  Each  lesson  contained  a Lesson  Sheet,  a Practical  Exer- 
cise Sheet,  a Practical  Exercise  Answer  Key,  a Criterion  Exercise 
Sheet,  and  a Criterion  Exercise  Answer  Key. 

The  Lesson  Sheet  was  divided  into  three  sections:  Objective, 
Purpose,  and  Concept.  The  Objective  section  stated  what  the  operator 
should  be  able  to  do  at  the  end  of  the  lesson,  the  Purpose  section 
stated  the  importance  of  the  lesson,  and  the  Concept  section  contained 
the  instructional  materials  with  examples.  A sample  lesson  sheet  is 
shown  in  Appendix  A. 

The  Practical  Exercise  Question  Sheet  contained  questions  on  the 
material  discussed  on  the  Lesson  Sheets.  These  questions  were  in- 
tended to  allow  the  operator  to  test  his  mastery  of  the  concepts  in 
the  lesson. 
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The  Practical  Exercise  Answer  Key  provided  the  operator  with  the 
correct  answers  to  the  questions  and  allowed  him  to  check  his  own 
knowledge.  The  operator  could  restudy  the  lesson  sheet  if  he  did  not 
do  satisfactorily. 

The  Criterion  Exercise  Sheet  contained  questions  relatinq  to  in- 
formation on  the  Lesson  Sheet.  These  questions  tested  the  same  thing 
as  the  practical  exercise  questions  and  were  the  final  check  for  mas- 
tery before  going  on  to  the  next  lesson. 

The  Criterion  Exercise  Answer  Key  enabled  the  training  monitor  to 
evaluate  the  operator's  performance  on  the  Criterion  Exercise.  If  the 
operator  performed  within  preestablished  criterion  levels,  he  or  she 
was  given  the  next  Lesson  Sheet.  If  not,  he/she  was  sent  back  for 
additional  review  or  to  an  instructor  for  individualized  help. 

In  this  training  system,  there  were  three  test  monitors  and  one 
subject  matter  expert.  They  helped  the  operators  and  monitored  their 
progress  through  the  system.  For  normal  usaqe  in  the  field  the  material 
could  be  bound  into  a booklet  and  monitoring  would  require  only  one 
person  (3,  below)  s 

1.  One  test  monitor  was  at  Station  1 to  control  handout  of  the 
Lesson  Sheet  and  score  the  Criterion  Exercises. 

2.  Two  test  monitors  were  at  Station  2 to  control  all  activities 
having  to  do  with  the  Practical  Exercises. 

3.  The  subject  matter  expert  was  at  Station  3 to  provide  subject 
matter  assistance  whenever  it  was  needed. 

The  operators  were  required  to  follow  five  basic  steps  to  complete 
each  lesson: 

1.  Read  the  Lesson  Sheet  for  comprehension. 

2.  Answer  the  Practical  Exercise  Questions. 

3.  Check  the  answers  with  the  Answer  Key  and  reread  the  Lesson 
Sheet  if  necessary. 

4.  Answer  the  Criterion  Exercise  Questions. 

5.  Have  the  answers  checked  with  the  test  monitor.  If  the  cri- 
terion was  met,  the  operator  was  allowed  to  progress  to  a new 
lesson;  if  the  criterion  was  not  met,  the  operator  was  asked 
to  review  the  Lesson  Sheet  and  Practical  Exercises  and/or  con- 
fer with  the  subject  matter  expert  and  then  rework  the  Cri- 
terion Exercise. 
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Development  of  the  Test  Scenarios 


In  order  to  assess  objectively  the  performance  of  the  operators, 
scenarios  were  developed  from  scenarios  used  in  a previous  study 
(Martinek,  Pilette,  and  Bigqs,  1978)  . These  were  developed  from  mag- 
netic tapes  of  typical  activations  produced  by  personnel,  tracked  and 
wheeled  vehicles,  and  typical  noise  such  as  artillery  and  aircraft 
collected  during  Project  1030  and  BASS  III  tests  conducted  at  the 
Modern  Army  Selected  Systems  Test  and  Evaluation  (MASSTER)  project. 
Fort  Hood,  Tex.  (U.S.  Army,  1970  and  1972) . Because  these  were  con- 
trolled exercises,  target  location  and  time  were  known  and  could  be 
related  to  sensor  activations. 

Eiqht  30-minute  "high-noise"  segments  were  selected:  four  seg- 
ments for  each  of  two  scenarios.  These  scenarios  were  matched  with 
respect  to  string  size  and  target  activity.  Two  of  the  seqments  in 
each  scenario  contained  three  targets  each  (low  activity)  and  two 
contained  six  targets  each  (high  activity) . Sensor  string  size  was 
systematically  varied,  i.e.,  in  the  three- target  case,  one  target  in- 
volved a 2-sensor  string,  one  involved  a 3-sensor  string,  and  one  in- 
volved a 4-sensor  string.  In  the  six- target  case,  these  were  doubled. 
Targets  were  selectively  arranged  within  each  30-minute  period  to 
provide  variation  and  balance. 

High-noise  means  that  the  segments  simulated  a battlefield  en- 
vironment containing  typical  activations  caused  by  fixed-wing  and 
helicopter  activity,  live  artillery  firing,  weather  activity,  and 
unreliable  (noisy)  sensors.  These  activations  were  not  eliminated 
when  they  occurred  with  the  original  target  patterns  and,  in  many 
cases,  were  added  to  scenario  segments  to  obtain  a reasonable  balance 
in  amount  and  type  across  all  the  segments. 


Training  Validation  Design 

The  effectiveness  of  the  training  materials  was  assessed  by  means 
of  a pretest/posttest  comparison  of  UGS  monitor  performance  on  opera- 
tionally realistic  sets  of  scenarios  using  simulated  R0376  event  re- 
corders. The  effectiveness  of  the  job  aid  (nomograph)  was  assessed 
by  a pretest/posttest  comparison  of  10  problems  requiring  the  opera- 
tors to  derive  target  speed  and  quantity  from  typical  activation 
patterns . 

Independent  Variables.  The  independent  variables  planned  for 
analysis  are  defined  below. 

1.  Sessions — Pretest  versus  posttest.  The  pretest  was  adminis- 
tered durinq  Session  I and  the  posttest  was  administered 
after  training  (Session  II)  . 
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2.  Sensor  String  Size — TVo-sensor  strinqs  versus  3-sensor  strings 
versus  4-sensor  strinqs. 

3.  Target  Activity — Low  tarqet  activity  (3  per  30  minutes)  versus 
high  tarqet  activity  (6  per  30  minutes) . A tarqet  was  defined 
as  one  or  more  vehicle (s)  or  person(s)  traveling  in  a column 
on  a trail. 

4.  Scenarios — TVo  2-hour  scenarios  were  matched  with  respect  to 
string  size  and  target  activity.  They  were  administered  in 

a counterbalanced  order  as  either  pretest  or  posttest.  Equiv- 
alent scenarios  were  not  quaranteed  because  different  amounts 
of  noise  were  present  and  different  targets  were  used  in  each 
scenario.  The  scenario  variable  was  of  interest  mainly  to 
increase  the  sensitivity  of  the  statistical  analysis  and  pro- 
vide data  for  future  scenario  development. 

5.  Groups — The  20  operators  were  randomly  assiqned  to  two  groups 
to  control  for  scenario  effects  by  counterbalancing.  Thus, 
one  group  received  scenario  "A"  durinq  the  pretest  and  the 
other  qroup  received  scenario  "B";  the  reverse  was  used  for 
the  posttest. 

The  independent  variables  were  analyzed  by  using  a mixed-model 
analysis  of  variance  design.  By  this  design,  individual  differences 
are  controlled  across  sessions  (training  effects)  and  all  important 
variables  analyzed  are  within-sub ject  comparisons.  The  three  factors 
of  groups,  sessions,  and  scenarios  were  counterbalanced  in  a Latin 
square,  and  as  a result,  interactions  of  these  factors  could  not  be 
obtained.  While  the  main  effect  of  sessions  alone  would  indicate  the 
effectiveness  of  the  self-paced  traininq  materials,  the  main  effects 
of  sensor  string  size,  target  activity,  and  the  higher  order  inter- 
actions were  included  in  the  detection  and  identification  analysis 
mainly  to  determine  the  specific  areas  of  usefulness. 

In  a previous  study,  it  was  shown  that  UGS  operator  detection  and 
identification  rights  were  significantly  lower  with  2-sensor  strings 
than  with  3-  or  4-sensor  strings  (Martinek,  Pilette,  and  Biqqs,  1978)  . 

In  addition,  percentage  completeness  on  hiqh-taroet-activity  scenario 
segments  was  significantly  lower  than  on  l^w- target-activity  scenario 
segments.  The  self-paced  training  materials  were  developed  to  help 
improve  performance  with  the  2-sensor  strinq  condition.  The  overall 
effect  of  the  traininq  should  also  help  to  improve  performance  in  the 
high-target-jetivity  condition  more  than  in  the  low-activity  condition, 
because  there  is  more  room  for  improvement  and  more  weiqht  was  given  to 
errors  in  the  high-activity  condition  in  the  determination  of  problem 
areas.  Thus,  interaction  effects  with  these  variables  and  sessions 
were  expected. 

Dependent  Variables.  For  the  assessment  of  the  training  materials, 
nine  separate  dependent  measures  were  planned.  The  first  three  dealt 
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with  the  overall  measures  of  performance;  the  last  six  were  related  to 
special  aspects  of  the  operator's  job. 

Using  the  independent  variables  listed  above,  separate  analyses 
of  variance  were  planned  for  the  following  dependent  variables; 

Detection  rights — If  an  operator  reported  an  activation  pattern 
on  the  display  that  was  a valid  target,  the 
response  was  classified  as  a detection  right. 
This  variable  is  perfectly  correlated  (for 
most  variables)  with  detection  completeness 
because  completeness  is  a ratio  of  the  targets 
detected  to  the  total  number  of  targets  pres- 
ent. Percentage  completeness  figures  are  given 
where  they  have  special  meaning. 

Wrongs  (false  — If  an  operator  reported  an  activation  pattern 
alarms)  on  the  display  that  was  not  a valid  target, 

the  response  was  classified  as  a false  alarm. 

Identification  — If,  in  addition  to  detecting  a valid  target, 
rights  the  operator  classified  it  correctly  by  type 

(personnel  or  vehicle) , the  response  was  also 
classified  as  an  identification  right. 

The  following  four  dependent  measures  relate  to  the  ability  to  use 
the  formulas  in  arriving  at  additional  intelligence  information  once  a 
target  has  been  detected. 

TT^  — Total  time  (in  minutes)  of  the  first  sensor 

activation  pattern  of  a sensor  string. 

Tm  — Time  difference  (in  minutes)  between  the  mid- 

point of  the  first  sensor  activation  pattern 
and  the  midpoint  of  the  second  sensor  activa 
tion  pattern. 

Speed  — Estimated  speed  (meters/minutes)  with  which 

the  target  moved  through  the  sensor  string. 

Speed  is  usually  calculated  from  time  (T  ) 
and  the  distance  between  two  sensors . 

Quantity  — Number  of  targets  estimated  to  be  in  the  tar- 

get column;  obtained  by  dividing  the  total 
length  of  the  column  by  the  estimated  distance 
between  targets. 

The  final  two  dependent  measures  were 

Reporting  time  — Time  between  target  detection  and  completion 
of.  the  report  as  written  by  the  operator. 
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Confidence 
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— The  operator's  estimates  on  a four-unit  scale 
as  to  how  confident  he  or  she  is  that  the 
activation  pattern  he/she  is  reporting  on  is 
a valid  tarqet. 

Job  Aid  (Nomograph)  . The  evaluation  of  the  -job  aid  involved  oivina 
paper-and- pencil  tests  in  which  10  tarqet  activation  problems  were  pre- 
sented on  the  pretest  and  posttest.  The  operator  did  not  detect  targets 
but  only  made  the  necessary  measurements  and  calculations  necessary  for 
reportinq  additional  intelligence  information  (speed  and  quantity) . 

The  posttest  was  prepared  by  randomly  rearranging  the  targets  of  the 
pretest  and  chanqinq  the  numerical  values  of  the  distances  between  the 
sensors  and  sensor  detection  ranqes.  Performance  was  compared  on  the 
following  dependent  measures:  TTj , T^,  speed  of  tarqet,  target  type, 
lenqth  of  column,  quantity,  and  reporting  time. 

Criteria  for  Scoring  Formula  Dependent  Variables.  The  intervals 
of  correctness  were  established  by  selecting  and  settinq  values  above 
and  below  the  established  school  solutions  for  TT^  and  TM.  The  crite- 
rion for  accepting  TTi  and  TM  values  for  the  individualized  training 
was  within  2/10  of  a minute  above  and  below  the  established  school 
solution.  The  result  of  accepting  a range  of  values  from  TM  intro- 
duced a range  of  values  for  speed.  Likewise,  accepting  a range  of 
values  for  TT^  introduced  a range  of  values  for  length  of  column  and 
quantity.  It  was  necessary  to  compute  ranqes  for  each  target  for 
scorinq  purposes. 

The  objective  of  the  job  aid  evaluation  was  to  assess  the  worth 
of  the  nomoqraph/UGS  ruler  combination,  usinq  the  hiqhest  standards 
that  could  be  needed  in  the  field.  The  range  of  acceptance,  therefore, 
for  the  pretest  and  posttest  of  the  job  aids  was  set  at  1/10  minute 
above  and  below  the  established  school  solution  of  TT^  and  TM. 

Test  Procedure 

The  test  plan  covered  a 4-day  period,  as  shown  in  Figure  2.  Pur- 
inq  the  first  day,  10  operators  received  test  procedure  training  plus 
the  pretest  in  the  morning:  10  received  the  same  in  the  afternoon. 

This  procedure  was  necessary  because  only  10  R0376  simulators  were 
available.  For  both  the  morning  and  afternoon,  five  operators  re- 
ceived Scenario  A and  five  received  Scenario  B for  the  pretest. 

During  the  second  day,  all  20  operators  received  the  eight  les- 
sons of  individualized  traininq.  This  training  did  not  require  the 
use  of  the  drive  mechanisms.  At  the  completion  of  the  training,  all 
operators  completed  a course  evaluation  questionnaire.  The  question- 
naire provided  a measure  of  operator  acceptance  of  the  materials  and 
the  traininq  system.  In  addition,  this  information  and  the  operator 
course  performance  data  provided  input  for  future  revisions  of  the 
course  material. 
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Day  1 


Test  procedure  and  pretest 


7:30  a.m.  - 9:30 

10:00  - 12:00 


1:00  - 3:00 

3:30  - 5:30 

Day  2 

8:40  a.m.  - 12:00 

1:00  - 4:00 

4:00  - 5:00 


Day  3 

9:00  a.m.  - 11:00 


2:00 

- 4:00 

Li 

7:30  a.m. 

- 8:30 

9:00 

- 11:00 

1:00 

- 3:00 

Traininq  on  test  procedure  with  the  R0376 
simulator  for  10  operators. 

Pretest — Each  operator  interpreted  18  tarqets 
in  a 2-hour  scenario  usinq  the  R0376  event 
recorder  simulator.  Five  operators  received 
Scenario  A and  five  operators  received  Sce- 
nario B . 

(Same  as  7:30  - 9:30) 

(Same  as  10:00  - 12:00) 

Individualized,  self-paced  traininq 

Four  lessons — hard  copy — 20  subjects 

Four  lessons — hard  copy — 20  subjects 

Student  traininq  evaluation  questionnaire — 

20  operators 

Posttest 

Posttest — Each  operator  interpreted  18  tarqets 
in  a 2-hour  scenario  usinq  the  R0376  event 
recorder  simulator.  Five  operators  received 
Scenario  A and  five  received  Scenario  B. 
Operators  and  scenarios  were  switched  from 
the  pretest. 

Same  as  above  for  second  qroup 
Nomoqraph,  pretest,  traininq,  and  posttest 
Pretest — hard  copy  format — 20  operators 
Traininq — lecture/discussion — 20  operators 
Posttest — hard  copy  format — 20  operators 


Fiqure  2.  Test  administration  schedule. 
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The  posttest  was  administered  on  the  third  day.  Durinq  the  morn- 
ing, the  five  operators  who  received  Scenario  A as  the  pretest  now 
received  Scenario  B,  and  those  who  initially  received  Scenario  B now 
worked  through  Scenario  A.  The  same  procedure  was  followed  in  the 
afternoon  with  the  remaining  10  operators. 

During  the  final  day,  all  20  operators  received  a pretest  in  the 
morning,  training  on  column- length  formula  calculations  using  the 
nomograph,  and  a posttest  in  the  afternoon. 

For  the  individualized  training.  Appendix  B presents  the  monitor 
instructions  for  the  (1)  initial  test  procedure  training,  (2)  pretest 
and  posttest  training,  and  (3)  actual  training  administration  procedure. 
The  job  aid  (nomoqraph)  monitor  instructions  are  not  provided  because 
they  were  given  in  a spontaneous  lecture-discussion  approach.  For  the 
nomograph  traininq,  a series  of  practical  problems  was  presented  to 
and  solved  by  the  class. 


RESULTS  AND  DISCUSSION 
Overall  Performance  Measures 

Detection  Rights.  The  followinq  variables  and  interactions  are  of 
primary  importance  and  are  discussed  in  the  order  given:  sessions, 
string  size,  sessions  x strinq  size,  target  activity,  and  sessions  x 
target  activity.  Table  1 presents  the  traininq  experiment  analysis 
of  variance  results  for  detection  rights.  Of  most  interest  is  the 
significant  difference  (.01  level)  found  for  the  sessions  variable. 

Out  of  a possible  18  rights,  the  average  operator's  correctness  for 
the  pretest  (Session  I)  was  10.4  rights  while  that  for  the  posttest 
(Session  II)  was  12.3  rights.  This  represents  an  18%  improvement  or 
about  one  target  per  operator  per  hour  and  demonstrates  the  effective- 
ness of  the  training.  Appendix  C presents  a discussion  of  why  this 
increase  is  attributable  to  training  as  opposed  to  a practice  effect. 

The  string-size  variable  (Table  1)  is  significant  at  the  .01  level. 
Out  of  a possible  6 rights  for  each  string  size,  the  2-sensor  strinqs 
resulted  in  2.6  rights,  the  3-sensor  strings  resulted  in  4.7  rights, 
and  the  4-sensor  strings  resulted  in  4.1  rights.  The  significantly 
fewer  target  detections  when  usinq  2-sensor  strinqs  aqrees  with  the 
results  of  a previous  study  (Martinek,  Pilette,  and  Biqqs,  1974). 

The  interaction  of  sessions  and  sensor  strinq  size  is  nonsignifi- 
cant, suggesting  that  the  increases  in  performance  for  the  three  string 
sizes  (from  Session  I to  Session  II)  are  similar.  However,  individual 
statistical  tests  conducted  on  each  sensor  strinq  size  for  differences 
between  sessions  indicates  that  this  condition  is  true  for  the  3-sensor 
and  4-sensor  string  sizes  but  not  the  2-sensor  strinq  size  (t  = 2.61, 
df  = 19,  p = .05)  . The  average  scores  for  this  interaction  are  pre- 
sented in  Table  2. 
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Table  1 

Analysis  of  Variance  Table  for  Detection  Riqhts 


Source 

df 

SS 

MSS 

F ratio 

Sig. 

level 

Between-sub jects 

19 

30.184 

Groups 

1 

.417 

.417 

.252 

NS 

Error 

18 

29.76"' 

1.654 

Within- sub jects 

220 

213.000 

1 

Sessions 

1 

6.0166 

6.017 

16.395 

.01 

String  size 

2 

45.0333 

22.517 

61.354 

.01 

Session  x strinq  size 

2 

1.2334 

0.6167 

1.680 

NS 

Target  activity 

1 

48.600 

48.600 

132.425 

.01 

Sessions  x target 

activity 

1 

1.6667 

1.667 

4.542 

.05 

String  size 

x target  activity 

2 

12.7000 

6.350 

17.302 

.01 

Scenario 

1 

12.1500 

12.150 

33.106 

.01 

Scenario  x target 

activity 

1 

5.4000 

5.400 

14.714 

.01 

Scenario  x string  size 

2 

0 

0.000 

0.000 

NS 

Scenario  x string  size 

x target  activity 

2 

5.200 

2.600 

7.084 

.01 

Session  x string  size 

x target  activity 

2 

0.233 

0.116 

0.316 

NS 

String  size  x groups 

2 

1 .7334 

0.867 

2.362 

NS 

Target  activity 

x groups 

1 

0.000 

0.000 

0.000 

NS 

String  size  x target 

activity  x groups 

2 

0.4000 

0.200 

0.545 

NS 

Error  (w) 

198 

72.633 

0.367 

Total 


239  243.184 


Table  2 
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Mean  Number  of  Detection  Riqhts  by  Session  and 
Sensor  String  Size 


2-sensor 

3-sensor  4-sensor 

Sum 

Session  I 

2.1 

4.5  3.8 

10.4 

Session  II 

3.1 

CD 

** 

12.3 

Average 

2.6 

4.7  4.1 

One  of  the  objectives  of  the  traininq  was  not  only  to  increase  the 
number  of  targets  detected  across  all  the  strinq  sizes,  but  especially 
to  increase  the  number  of  targets  detected  with  2-sensor  strinqs.  As 
shown  in  Table  2,  the  number  of  targets  detected  with  2-sensor  strinqs 
did  increase  from  2.1  to  3.1.  The  latter  value,  however,  is  still  well 
below  the  corresponding  figures  for  the  3-  and  4-sensor  strings  (4.8 
and  4.4  respectively)  indicating  that  this  objective  was  not  completely 
satisfied.  One  obvious  possibility  s that  2-sensor  strinqs  are  more 
difficult  to  interpret.  This  result  reinforces  the  validity  of  the 
doctrine  calling  for  three  or  more  sensors  in  a strinq. 

The  tarqet  activity  variable  is  significant;  each  operator  de- 
tected, on  the  averaqe,  4.4  targets  during  low  target  activity  and  7.1 
during  high  activity.  These  figures  are  based  on  a possible  total  of 
6 targets  for  the  low-activity  and  12  for  the  high-activity  condition. 
Interpretation  of  all  experimental  effects  involving  the  target  activity 
variable  must  consider  that  twice  as  many  targets  are  detectable  in  the 
high- target- activity  condition  as  in  the  low.  Thus,  significant  dif- 
ference in  detection  rights  was  expected.  Ot  more  importance  is  the 
detection  completeness  of  targets  detected  in  each  activity  condition. 
During  low  target  activity,  72%  of  the  targets  were  detected  but  durinq 
high  target  activity  only  59%  were  detected,  a difference  which  is 
statistically  significant  (t  =»  4.94,  df  = 19,  p < .01). 

The  interaction  of  sessions  and  tarqet  activity  is  significant, 
which  suqqests  that  the  increases  in  performance  for  the  two  target 
activity  conditions  from  Ses jion  I to  Session  II  are  different.  In- 
dividual statistical  tests  indicate  that  the  increase  in  performance 
is  attributable  to  the  high-target-activity  condition  (t  «•  2.18, 
df  ■ 19,  p < .05).  The  raw  score  values  for  this  interaction  are 
presented  in  Table  3,  along  with  completeness  scores  reported  in 
parentheses . 
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Table  3 


Mean  Number  of  Detection  Rights  and  Completeness  by 
Session  and  Target  Activity 


Low  target  activity 

High  target  activity 

Sum 

Session  I 

4.1  (68%) 

6.3  (53%) 

10.4 

Session  II 

4.6  (76%) 

7.8  (65%) 

12.3 

Average 

4.4 

7.1 

One  of  the  objectives  of  the  training  was  to  increase  the  complete- 
ness of  targets  detected,  especially  during  high  target  activity.  As 
shown  in  Table  3,  the  significant  completeness  increase  from  53%  to  65% 
satisfied  this  objective.  This  latter  figure,  however,  is  still  below 
the  corresponding  figure  for  the  low-activity  condition  (76%) . These 
results  indicate  that  the  training  was  effective  in  increasing  per- 
formance, predominantly  in  the  high- target-activity  condition. 

The  string  size  by  target  activity  interaction  presented  in  Table  4 
is  significant  at  the  .01  level.  Thus,  although  there  are  large  dif- 
ferences in  the  mean  number  of  detection  rights  in  the  3-  and  4-sensor 
strings,  there  is  only  a small  difference  in  the  2-sensor  string.  When 
these  figures  are  corrected  for  the  different  numbers  of  targets  in  the 
high-  and  low-activity  conditions,  i.e.,  completeness,  the  effect  on 
operator  performance  is  clearer.  As  first  observed  in  a previous  study 
(Martinek,  Pilette,  and  Biggs,  1974) , completeness  is  lower  for  all 
sensor  string  sizes  in  the  high-activity  condition,  but  the  significant 
interaction  indicates  that,  as  the  number  of  sensors  in  the  string  be- 
comes smaller  (i.e.,  from  4 to  2) , it  becomes  more  difficult  for  a 
monitor  to  detect  targets  during  high  target  activity  as  opposed  to  a 
low  target  activity.  For  the  4-sensor  condition,  a 4%  difference 
exists  between  the  low-  and  high-activity  conditions?  for  the  3-sensor 
condition,  the  difference  increases  to  13%,  and  for  the  2-sensor  condi- 
tion the  difference  increases  to  23%. 

These  results  should  be  interpreted  with  caution.  The  poorer  per- 
formance associated  with  the  2-sensor  string  condition  may  be  due  to  a 
combination  of  a disproportionate  personnel/ vehicular  target  ratio  and 
various  conditions  prevailing  at  the  time  the  target  activation  data 
were  collected.  In  general,  personnel  do  not  produce  as  good  an  acti- 
vation pattern  as  do  vehicles.  The  target  types  for  the  2-sensor  string 
condition  were  5 personnel,  6 vehicles,  and  1 combination;  for  the  3- 
sensor  string  condition  target  types  were  3 personnel,  8 vehicles,  and 
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Table  4 

Mean  Number  of  Detection  Riqhts  and  Completeness  by 
Strinq  Size  and  Tarqet  Activity 


2-sensor  3- sensor  4-sensor  Sum 

Low- tarqet  activity  2.4  (59%)  3.5  (86%)  2.9  (71%)  8.7 

High- tarqet  activity  2.9  (36%)  5.9  (73%)  5.4  (67%)  14.0 


1 combination;  and  for  the  4-sensor  strinq  condition  tarqet  types  were 

2 personnel  and  9 vehicles.  In  addition,  because  all  personnel  tarqet 
activations  were  collected  durinq  the  BASS  III  tests  (U.S.  Army,  1970) 
and  all  vehicular  tarqet  activations  were  qathered  durinq  the  Troiect 
1030  tests  (11. S.  Army,  1972)  with  a 2-year  interval  separatinq  these 
exercises,  differences  in  equipment  reliability,  implantation  techniques, 
qround  location,  weather,  and  soil  type  could  have  had  a marked  influ- 
ence on  tarqet  presentation  quality  and  hence  upon  operator  per formance . 
Thus,  the  differences  found  for  strinq  size  may  be  due  to  differences 

in  the  level  of  tarqet  difficulty. 

The  scenario  variable  is  siqnificant  at  the  .01  level,  which  indi- 
cates that  Scenario  A,  with  an  averaqe  of  5 riqhts,  was  more  difficult 
than  Scenario  B,  with  an  averaqe  of  6.4  riqhts.  Moreover,  the  siqnifi- 
cant interaction  between  scenario  and  target  activity  ( .01  level)  in- 
dicates noticeably  lower  performance  (5.9)  in  the  hiqh-tarqet-activity 
condition  of  Scenario  A (see  Table  5) . Althouqh  not  important  to  the 
obiectivos  of  this  evaluation,  a thorouqh  analysis  of  the  causes  of 
this  outcome  could  be  useful  for  future  scenario  development. 

The  triple  interaction  of  strinq  size,  scenarios,  and  tarqet  activ- 
ity is  siqnificant  at  the  .01  level.  The  mean  number  of  riqhts  by  con- 
dition is  presented  in  Table  6. 

Table  6 reveals  more  detailed  information  concerninq  siqnificant 
scenario  differences.  The  scenario  and  target  activity  interaction 
seen  in  Table  5 indicate  that  the  hiqh-activi ty  condition  of  Scenario 
A contained  more  di fficult-to-detect  targets.  This  is  especially  true 
of  the  2-sensor  string  condition  within  the  high-activity  condition  of 
Scenario  A.  However,  this  conclusion  and  others  which  can  be  drawn  do 
not  have  a direct  bearing  on  the  objectives  of  the  study  and  are  im- 
portant only  for  future  scenario  development . All  the  other  interac- 
tions tested  wore  found  to  be  not  significant. 


Table  5 

Mean  Number  of  Detection  Rights  by  Scenario 
and  Target  Activity 


Low- target  activity 
High- target  activity 


Average 


Table  6 

Mean  Number  of  Detection  Rights  by  String  Size, 
Scenario,  and  Target.  Activity 


Low  target  activit 


Scenario  A Scenario  B 


High  target  activity 


Scenario  A Scenario  B 


String  size 


2-sensor 


3- sensor 


4- sensor 


Average 


Wrongs  (False  Alarms) . Of  major  interest  to  this  study  is  the 
small  number  of  false  alarms:  only  three  occurred  during  Session  I and 
seven  during  Session  II,  too  few  for  conducting  an  analysis  of  variance. 
The  false  alarms  were  distributed  in  a random  fashioip  across  eight 
operators.  Because  of  the  randomness  with  which  the  errors  were 
committed,  it  is  not  possible  to  draw  generalizations  concerning  the 
causes  for  false  alarms. 

The  small  number  of  false  alarms  that  occurred  in  this  study  repre- 
sents a sharp  contrast  to  \he  large  number  of  false  alarms  reported  in 
a multidisplay  comparison  study  using  similar  scenarios  (Martinek, 
Hilligos,  and  Lavicka,  1978)  . In  the  present  study  the  average  number 
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of  false  alarms  per  2-hour  scenario  was  .2  while  that  for  the  RC376 
from  the  multidisplay  study  was  5.6.  Such  a large  difference,  even 
though  similar  scenarios  were  used,  could  be  caused  by  differences  in 
(1)  instructions  (set)  or  context  of  the  study  (comparison  of  displays 
versus  traininq) , and/or  (2)  school  orientation  on  accuracy  of  tarqet 
reportinq.  The  latter  reason  is  probably  more  siqnificant,  because 
the  averaqe  operator  in  this  study  had  qraduated  (several  months  since) 
from  the  U.S.  Army  DCS  school.  The  Army  school  typically  uses  classi- 
cal patterns  (stairstep  shapes)  for  practical  exercises,  and  this  may 
automatically  inqrain  accurate  tarqet- reportinq  habits  amonq  the 
trainees.  They  are  less  likely  to  make  errors  of  commission,  but 
they  may  be  more  likely  to  make  errors  of  omission. 

The  multidisplay  study  used  12  Navy  personnel  who  had  been  trained 
at  another  URS  school  but,  perhaps  more  important,  had  been  on  the  iob 
for  several  years  and  had  received  field  experience  on  several  maneuvers. 
These  operators  committed  more  errors  of  commission,  but  they  also  de- 
tected more  tarqets  on  the  averaqe,  which  may  reflect  the  Navy's  em- 
phasis on  completeness.  . 

Identification  Rights.  Table  7 presents  the  analysis  of  variance 
results  for  identification  riqhts.  Of  most  interest  is  the  siqnificant 
difference  (.01  level)  for  the  sessions  variable.  The  averaqe  opera- 
tor's identification  riqhts  score  for  the  pretest  session  was  8.3  while 
that  for  the  posttest  session  was  10.9.  This  represents  an  averaqe  im- 
provement for  each  operator  of  2.55  tarqets  or  a 31%  gain  over  pretest 
performance.  This  is  a qreater  increase  than  would  be  expected  from 
the  detection  results,  which  showed  an  18%  improvement.  The  identifi- 
cation improvement  of  31%  seems  to  indicate  that  the  traininq  was  more 
effective  in  improvinq  target  identification  than  it  was  for  improving 
tarqet  detection,  because  detection  improvement  was  only  18%.  However, 
identi fication  completeness  (the  ratio  of  tarqets  identified  to  those 
detected)  indicates  that  there  are  no  siqnificant  differences  between 
Session  I and  Session  II. 

The  string-size  variable  is  siqnificant.  Table  8 shows  that  with 
6 tarqets  possible  for  each  string  size,  operators  identified  2.2  of 
the  tarqets  activating  the  2-sensor  strinqs,  3.9  of  those  activating 
the  3-sensor  strinqs,  and  3.5  of  those  activating  the  4-sensor  strinqs. 

The  interaction  of  sessions  by  strinq  size  in  Table  8 is  nonsignifi- 
cant, which  suggests  that  the  increases  in  performance  from  Session  I 
to  Session  II  for  the  three  strinq  sizes  are  similar. 
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Table  7 
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Analysis  of  Variance  Table  for  Identification  Rights 


Source 

df 

SS 

MSS 

F ratio 

Sig. 

level 

Bet ween- sub  j ec  ts 

19 

34.7125 

Groups 

1 

7.7041 

7.7041 

5.1344 

.05 

Error 

18 

27.0084 

1.5005 

Within- subjects 

Session 

1 

10.8375 

10.8375 

25.2740 

.01 

Strinq  size 

2 

34.8083 

17.4042 

40.5882 

.01 

Session  x string  size 

2 

0.9250 

0.4625 

1.0786 

NS 

Tarqet  activity 

1 

22.2041 

22.2041 

51.7820 

.01 

Session  x tarqet 

activity 

1 

1.8375 

1.8375 

4.2852 

.05 

String  size  x target 

activity 

2 

6.3084 

3.1542 

7.3559 

.01 

Scenario 

1 

5.1041 

5.1041 

11.9032 

.01 

Scenario  x target 

activity 

1 

2.6043 

2.6043 

6.0735 

.05 

Scenario  x string  size 

2 

0.8584 

0.4292 

1 . 0009 

NS 

Scenario  x string  size 

x target  activity 

2 

4.7582 

2.3791 

5.5483 

.01 

Sessions  x string  size 

x target  activity 

2 

0.0250 

0.0125 

0.0292 

NS 

String  size  x group 

2 

11.4084 

5.7042 

13.3027 

.01 

Target  activity 

x groups 

1 

1.2043 

1.2043 

2.8085 

NS 

String  size  x target 

activity  x groups 

2 

1.3082 

0.6541 

1.5254 

NS 

Error  (w) 

198 

84.8916 

0.4288 

Total 

239 

222.796 

20 


Table  8 

Mean  Number  of  Identification  Rights  by 
Session  and  String  Size 


2-sensor 

3-sensor 

4-sensor 

Sum 

Session  I 

1.7 

3.7 

3.0 

8.3 

Session  II 

2.7 

4.2 

4.0 

10.9 

Average 

2.2 

3.9 

3.5 

Identification  completeness  (i.e.,  the  ratio  of  the  number  of  tar- 
gets identified  to  those  detected)  for  this  interaction  is  shown  in 
Table  9.  There  are  no  significant  differences  between  Session  I and 
Session  II.  Individual  t-tests  were  computed  for  each  string  size  to 
test  the  significance  of  the  difference  between  the  sessions  means. 

The  values  for  the  2-,  3-,  and  4-sensor  strings  were  1.66,  1.07,  and 
1.67  respectively  (n  *=  9) . The  results  indicate  that  once  a target 
was  detected,  the  sensor  string  size  was  immaterial:  roughly  80%  to 
90%  were  identified.  This  is  similar  to  the  previous  study  (Martinek, 
Hilligos,  and  Lavicka,  1978)  in  which  the  R0376  display  resulted  in 
90%  completeness.  For  both  the  Army  and  the  Navy  operators,  therefore, 
once  a target  has  been  detected,  the  probability  of  a correct  iden- 
tification is  similarly  high. 


Table  9 

Percentage  Identification  Completeness  by  Session 
and  String  Size 


2-sensor 

3-sensor 

4-sensor 

( 

Session  I 

72 

82 

78 

Session  II 

88 

87 

92 

The  tarqet  activity  variable  is  siqnificant.  As  with  the  riqhts 
(detection)  results,  this  outcome  was  expected  for  the  identification 
riqhts.  Of  the  tarqets  detected  in  the  low-tarqet-activity  condition, 
90%  were  identified,  whereas  in  the  hiqh-tarqet-activity  condition  81% 
were  identified. 

Of  more  importance  is  the  siqnificant  interaction  between  sessions 
and  tarqet  activity  (Table  7) . The  means  are  qiven  in  Table  10.  The 
siqnificant  interaction  suqqests  that  performance  did  not  increase 
similarly  for  both  activity  conditions  but  that  the  liiqh-activity  con- 
dition resulted  in  a larqer  increase.  The  traininq  objective  of  better 
performance  for  the  hiqh-activity  condition  was  satisfied. 


Table  10 

Mean  Number  of  Identification  Riqhts  by 
Session  and  Tarqet  Activity 


Low- tarqet  activity  Hiqh-tarqet  activity  Sum 


Session  I 


Session  II 


Averaqe 


Table  11  presents  identification  completeness  (i.e.,  percentaqe  of 
tarqets  identified  to  tarqets  detected) . There  are  no  siqnificant  dif- 
ferences between  Session  I and  Session  II.  The  t-test  values  (df  = 19) 
were  1.409  for  the  low-tarqet-activity  condition,  tnd  1.20  for  the  hiqh- 
tarqet-activity  condition. 

The  strinq  size  by  tarqet  activity  interaction  presented  in  Table 
12  is  siqnificant  at  the  .01  level.  This  table  reveals  that  the  number 
of  identification  riqhts  is  substantially  lower  for  the  2-sensor  strinq 
condition  for  both  the  low-  and  hiqh-tarqet-activity  conditions.  These 
results  are  very  similar  to  those  for  detection  riqhts. 
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Table  11 

Percentage  Identification  Riqhts  Completeness  by 
Session  and  Tarqet  Activity 


Low  tarqet  activity  Hiqh 

target  activity 

Session  I 

85 

79 

Session  II 

94 

85 

Table  12 

Mean  Number  of  Identification  Rights  by 
Strinq  Size  and  Target  Activity 

2-sensor  3-sensor 

4-sensor 

Low- target  activity 

2.0 

3.1 

2.7 

Hiqh-target  activity 

2.3 

4.8 

4.3 

Average 

2.1 

4.0 

3.5 

The  scenario  variable  was  significant  at  the  .01  level,  which  in- 
dicates that  Scenario  A (with  an  average  of  4.4  rights)  was  more  diffi- 
cult than  Scenario  R (with  an  average  of  5.3  rights) . The  difference 
in  scenarios  is  probably  due  to  a difference  in  target  difficulty,  as 
revealed  in  the  high-  and  low-target-activity  conditions  shown  in 
Table  13.  This  table  suggests  that  the  interaction  (significant  at  the 
.01  level)  is  due  to  the  noticeably  lower  performance  in  the  hioh-target 
activity  condition  (4.9  rights)  of  Scenario  A.  As  with  the  detection 
outcome,  a further  analysis  of  these  data  is  beyond  the  objectives  of 
this  research  but  could  be  useful  for  scenario  development  in  future 
research. 

The  triple  interaction  of  string  size,  scenario,  and  target  activ- 
ity is  significant.  The  results  are  presented  in  Table  14.  For  each 
target  activity  level,  note  the  lower  number  of  correct  identifications 
for  targets  Involving  2-sensor  strings,  and  the  differences  across 
scenarios  for  2-sensor  strings. 
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Table  13 


Mean  Number  of  Identification  Riqhts  by 
Scenario  and  Target  Activity 


Scenario  A 

Scenario  B 

Sum 

Low-tarqet  activity 

3.8 

4.0 

7.8 

Hiqh-tarqet  activity 

4.9 

6.5 

11.4 

Averaqe 

4.4 

5.3 

Table  14 

Mean  Number 

of  Identification  Rights  by  Strinq 
Scenario,  and  Tarqet  Activity 

Size, 

Low  tarqet  activity 

High  tarqet  activity 

Strinq  size  Scenario  A Scenario  B 

Scenario 

A Scenario  B 

2-sensor 

1.1  .9 

.7 

1.7 

3-sensor 

1.4  1.8 

2.3 

2.6 

4-sensor 

1.3  1.4 

2.1 

2.3 

Averaqe 

1.3  1.4 

1.7 

2.2 

The  interaction  of  sessions  x strinq  size  x tarqet  activity  was 
not  significant,  which  indicates  that  the  previous  discussions  of  these 
variables  need  not  be  modified  due  to  a triple  order  effect. 

The  remaininq  interactions  are  strinq  size  x scenario,  and  those 
involvinq  groups.  They  have  little  bearing  on  the  objectives  of  this 
study  and  are  therefore  not  discussed. 

Confidence  Ratings.  The  analysis  of  variance  results  for  the  con- 
fidence variable  is  presented  in  Table  15. 
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Table  15 


i 

I 


Analysis  of  Variance 

Table  for 

Confidence 

Values 

Source 

df 

SS 

MS 

F ratio 

Sig. 

level 

i 


Be tween- sub jects 

19 

6.61 

Groups 

1 

.90 

.90 

2.84 

NS  j 

Error  (b) 

18 

5.71 

.38 

Within-sub jects 

20 

1.25 

Sessions 

1 

.004 

.004 

.062 

NS 

Scenarios 

1 

.08 

.08 

1.25 

NS 

Error  (w) 

18 

1.17 

.065 

Total 

39 

7.86 

I 


No  significant  differences  occurred.  Either  there  is  no  difference 
in  confidence  or  there  are  differences  in  confidence  but  the  ratinq 
technique  for  measuring  an  operator's  confidence  was  inadequate.  The 
latter  case  is  probably  true,  since  the  variation  be tween-subjects 
(Table  15)  is  far  qreater  than  the  variation  within-sub jects . This 
occurred  because  each  operator  tended  to  select  and  use  only  a particu- 
lar confidence  level,  even  though  some  target  activation  patterns  were 
more  difficult  than  others. 

Measuring  the  confidence  of  operators  could  be  useful  to  the  field 
commander  when  he  is  weighing  fragments  of  intelligence  from  several 
sources  in  a battle  situation.  Knowing  how  confident  the  operator  is 
in  his  judgment  could  influence  the  importance  that  the  commander  will 
place  on  the  information.  This  was  the  first  time  that  these  monitors 
had  been  asked  to  rate  their  confidence.  The  rating  system  perhaps 
could  have  been  more  useful  if  the  operators  had  been  given  more  ex- 
tensive instruction  and  had  been  helped  to  develop  adequate  anchor 
points  across  the  scale. 

Reporting  Time.  The  F-max  test  of  the  homogeneity  of  variance 
indicated  that  this  basic  assumption  in  use  of  the  analysis  of  vari- 
ance model  was  not  satisfied.  Therefore,  a log  transformation  of  the 
reporting  time  scores  was  performed.  Table  16  presents  the  results 
of  the  analysis  of  variance,  using  the  log  transform  of  reporting  time. 
Both  the  groups  and  sessions  variables  were  significant  at  the  .05 
level.  The  results  on  groups  has  no  impact  on  the  objectives  of  the 
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study  except  for  control  purposes.  However,  the  results  on  the  ses- 
sions variable  indicates  that  the  training  resulted  in  a time  saving 
in  the  reporting  of  targets.  An  operator  required  5.9  minutes  to 
report  a target  before  training  and  5.0  minutes  after  training — a time 
saving  of  15%  for  each  target  reported.  Operationally  this  means  that 
a target  would  be  reported  1 minute  earlier.  In  addition,  during  an 
overload  situation  when  operators  have  more  targets  than  they  can 
report,  an  operator  who  has  received  special  training  will  be  able  to 
report  more  targets . 


Table  16 


Analysis 

of  Variance 

Table  for 

Reporting  Time3 

Source 

df 

SS 

MS 

F ratio 

P 

Be tween- sub jects 

19 

.4151 

Groups 

1 

.1780 

.1780 

13.59 

.01 

Error  (b) 

18 

.2371 

.0131 

within-sub jects 

20 

.2181 

Sessions 

1 

.0440 

.0440 

4.94 

.05 

Scenarios 

1 

.0127 

.0127 

1.43 

NS 

Error  (w) 

18 

.1612 

.0089 

Total 

39 

.6332 

a 


Log  transform  of  reporting  time  used. 


Additional  time  savings  (4  minutes)  could  also  be  incurred  by 
having  the  operators  send  out  a target  detection  report  immediately 
after  target  detection  and  then  by  a followup  target  identification 
report  several  minutes  later.  For  target  acquisition,  these  few  min- 
utes could  be  very  valuable  for  processing  the  information  and  alertina 
gun  crews . 


Measurement  and  Computational  Procedures 

Use  of  the  column-length  formula  provides  two  bits  of  information 
used  for  military  intelligence  purposes,  target  speed  and  target  quan- 
tity (the  number  of  targets  in  a column) . Solution  of  this  formula  re' 
quires  two  target- pat tern  measurements  which  are  taken  directly  from 
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the  activation  readout.  These  are  TT^  and  Tm,  which  have  been  dis- 
cussed previously.  Analyses  are  reported  below  for  TTj,  Tm,  tarqet 
■^peed,  and  tarqet  quantity.  The  school-solution  criteria  of  correct- 
ness are  discussed  in  the  Method  section. 

TTi . The  analysis  of  variance  for  TTj  is  presented  in  Table  17. 
The  sessions  variable  is  significant  at  the  .01  level,  which  indicates 
that  the  training  was  effective.  The  posttest  results  of  6.45  average 
correct  measurements  represents  a 63%  improvement  over  pretest  results 
of  3.95.  The  groups  and  scenarios  variables  are  nonsignificant. 


Table  17 

Analysis  of  Variance  Table  for  TT^  Values 


Source 

df 

SS 

MS 

F ratio 

Sig. 

level 

Be twee n- s ub j ec ts 

19 

109.40 

Groups 

1 

3.60 

3.60 

.613 

NS 

Error  (b) 

18 

105.80 

5.87 

Within-sub jects 

20 

184.7 

Sessions 

1 

62.5 

62.50 

9.27 

.01 

Scenarios 

1 

.9 

.90 

.134 

NS 

Error  (w) 

18 

121.3 

6.74 

Total 

39 

294.1 

Tm.  The  analysis  of  variance  for  Tm  values  is  presented  in  Table 
18.  The  sessions  variable  is  statistically  significant  at  the  .01  level 
which  indicates  that  the  training  was  effective.  Posttest  results  of 
6.8  average  correct  measurements  represent  a 58%  improvement  over  pre- 
test results  of  4.3.  The  groups  and  scenarios  are  nonsignificant. 

The  results  of  TT^  and  Tm  are  almost  identical.  This  was  not  unex- 
pected because  both  involve  measurements  taken  directly  from  the  X-T 
display  and,  therefore,  demonstrate  the  effectiveness  of  the  training 
given  for  the  endpoints  (Lesson  3)  and  the  use  of  the  UGS  ruler  iob 
aid. 
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Table  18 


Analysis  of  Variance  Table  for  T Values 

M 


Source 

df 

SS 

MS 

F ratio 

Sig. 

level 

Be tween- subjects 

19 

88.90 

Groups 

1 

0 

0 

0 

NS 

Error  (b) 

18 

88.90 

4.94 

Wi thin- sub jects 

20 

217.00 

Sessions 

1 

62.50 

62.50 

8.35 

.01 

Scenarios 

1 

19.60 

19.60 

2.62 

NS 

Error  (w) 

18 

134.90 

7.49 

Total 

39 

305.90 

Target  Speed.  Table  19  presents  the  results  of  the  analysis  of 
variance  for  the  target  speed  variable.  Again,  there  is  a significant 
(.01  level)  increase  of  posttest  results  over  pretest  results. 

The  average  operator  performance  on  the  pretest  is  3.9  correct  and 
that  for  the  posttest  is  6.4  correct,  representing  a 64%  increase  in 
accuracy  of  reporting  the  speed  of  the  targets.  However,  most  of  this 
increase  is  attributable  to  the  increased  accuracy  of  obtaining  T . 

The  groups  and  scenario  variables  are  nonsignificant. 

Quantity . Table  20  presents  the  results  of  the  analysis  of  vari- 
ance for  quantity.  Again  the  session  variable  is  significant  at  the 
.01  level.  The  average  operator  performance  on  the  pretest  was  3.95 
correct  and  that  for  the  posttest  was  6.45  correct,  a performance  in- 
crease of  63%.  The  difference  between  operator  performance  on  the  two 
scenarios  (4.5  correct  on  Scenario  A and  5.9  correct  on  Scenario  B) 
was  significant  at  the  .05  level. 


Within-Course  Performance 


In  developing  instructional  systems,  it  is  standard  procedure  to 
collect  data  that  can  assist  the  course  developer  to  effectively  revis 
his  materials.  In  addition  to  pretest  and  posttest  performance,  opera 
tor  performance  on  individual  lessons  can  provide  effective  guidelines 
for  revisions  that  can  result  in  improved  posttraining  performance  on 
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Table  19 

Analysis  of  Variance  Table  for  Tarqot  Speed  Values 


Source 

df 

SS 

MS 

F ratio 

Siq. 

level 

Be  tween- sub ]oc  ts 

19 

B2.10 

Groups 

1 

.40 

.40 

.09 

NS 

Error  (b) 

IB 

B2.70 

4.54 

Wl thin-sub locts 

20 

157.00 

Sessions 

1 

62.50 

62.50 

13.33 

.01 

Scenarios 

1 

10.00 

10.00 

2.13 

NS 

Error  (w) 

IB 

H4.50 

4.69 

Total 

39 

239.10 

Table  20 


Analysis 

of  Variance  Table  for  Quantity  Values 

Source 

df 

SS 

MS 

F ratio 

Siq. 

level 

Be twee n-sub locts 

19 

Groups 

1 

2.50 

2.50 

.34 

NS 

Error  (b) 

IB 

130.90 

7.27 

wi thin-sub jacts 

20 

151 .00 

Sessions 

1 

62.50 

62.50 

16.32 

.01 

Scenarios 

1 

19.60 

19.60 

5.12 

.05 

Error  (w) 

lfl 

6B.90 

3,83 

Total 

39 

2B4.40 

subsequent  administrations.  Althouqh  a formal  evaluation  usinq  opera- 
tor performance  data  was  not  within  the  scope  of  this  effort,  the  data 
on  individual  lesson  performance  are  provided,  to  present  a complete 
picture  of  the  effect  of  the  traininq  and  to  allow  for  improvements  of 
the  course  materials  in  subsequent  efforts. 

Table  21  presents  the  scores  on  the  criterion  exercise  for  each 
lesson  for  each  operator.  Each  operator's  first  attempt  is  qiven  as 
well  as  scores  on  subsequent  recycles.  It.  should  be  recalled  that  an 
operator  had  to  qet  100%  correct  on  each  criterion  exercise  before 
beinq  allowed  to  proceed  to  the  next  lesson.  It  is  apparent  from 
Table  21  that  Lessons  3 and  5 presented  the  qreatest  amount  of  diffi- 
culty for  the  operator.  Over  50%  of  the  operators  had  to  recycle  on 
these  two  lessons,  which  indicates  that  the  traininq  materials  should 
be  revised. 

I I 

As  with  most  individualized,  self-paced  traininq  systems,  opera- 
tors varied  widely  in  the  amount  of  time  they  required.  One  operator 
had  completed  half  of  the  lessons  by  10:15  a.m.,  and  over  50%  of  the 
operators  finished  Lesson  4 by  the  end  of  the  morninq.  One  operator 
completed  all  eiqht  lessons  by  2:30  p.m.,  (1-1/2  hours  early),  75% 
of  the  operators  were  finished  by  3:00  p.m.,  and  only  one  operator 
took  the  full  time  allotted. 


Student  Questionnaire  Results 

The  questionnaire  was  administered  to  obtain  \ measure  of  student 
acceptance  of  the  individualized,  self-paced  traininq  system  and  to 
provide  further  data  that  could  help  quide  revisions  of  the  traininq 
system  and  materials.  Summaries  of  the  questions  and  answers  are 
presented  below. 

Overall  Assessment  of  the  Traininq  System.  Only  2 of  the  20  opera- 
tors indicated  that  they  had  ever  received  like  traininq,  and  this  had 
occurred  in  hiqh  school.  When  asked  for  their  overall  impression  of 
the  traininq  compared  to  the  conventional  classroom  method,  13  indi- 
cated it  was  "definitely  better,"  and  7 indicated  it  was  "somewhat 
better.”  None  indicated  it  was  "worse"  or  "no  difference."  Nineteen 
operators  indicated  they  would  like  to  receive  more  traininq  usinq  this 
technique.  Overall,  the  subjects  were  quite  favorably  disposed  toward 
the  traininq  qiven. 


Best  and  Worst  Things  About  the  Training  System, 
answers  were  qiven  to  the  question  "What  did  you  like 
this  system  of  traininq?" 

• Tolerance  allowed  in  answers? 

• Open-minded  instructors; 


The  followinq 
the  most  about 
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Management  Sheet  of  Criterion  Exercise  Pecycling 


• Worked  at  own  pace; 

• Made  learning  a little  more  interesting; 

• Lessons  were  easier  to  understand; 

• Question  sheets  were  not  too  long; 

• Checking  own  answers; 

• Relaxing  atmosphere,  informality,  not  as  boring; 

• Refreshed  memory; 

• Lesson,  then  application  and  help  from  instructors; 

• Had  examples  on  paper  to  help  you,  whereas  in  the  classroom 
you  don't; 

• Absence  of  boring  lectures  and  excess  verbiage;  and 

• Learn  a little  more  because  you  get  tested  right  then. 

The  following  answers  were  given  to  the  question  "What  did  you 
dislike  most  about  this  system  of  traininq?" 

• Nothing; 

• Did  not  have  time  to  get  important  items  into  personal  notes 
for  reference; 

• Not  enough  breaks; 

• Too  much  tolerance  allowed  on  some  answers; 

• Some  of  material  was  incorrect; 

• Tired  at  the  end  of  the  last  couple  of  lessons,  and  you  don't 
always  grasp  the  full  understanding; 

• Some  of  the  material  was  not  presented  very  clearly — too  easy 
to  misunderstand  questions; 

• Too  much  time  waitinq  in  line  between  stations  to  get  your 
materials; 

• Not  enough  discussion; 

• Too  long,  too  many  tests  over  too  short  a period; 
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• Went  Into  too  much  detail  on  instructions  at  times;  and 

• The  feelinq  of  hurrying  to  get  done. 

Suggestions  for  Improvement  of  the  Training  System.  The  following 
answers  were  given  to  the  question  "If  you  could  make  some  changes  in 
how  this  training  system  operates  what  would  you  change?" 

• Use  better  equipment  (note ; two  machines  in  the  testing  phase 
caused  difficulty) ; 

• Give  10-minute  break  every  hour; 

• Increase  the  probability  that  the  student  actually  uses  the 
material,  perhaps  with  longer  or  more  complicated  test; 

• Leave  it  the  way  it  is; 

• Have  more  instructors  available  (to  reduce  waiting  in  line); 

• Give  out  the  lesson  and  practical  exercise  at  the  same  time; 

• Include  lecture  and  group  question  and  answer  period,  also 
hands-on  time  for  new  students; 

• Make  some  of  the  more  difficult  questions  more  simple;  and 

• Do  not  know. 

Student  Assessment  of  Specific  Lessons.  In  answer  to  "Were  there 
any  lessons  that  you  felt  you  did  not  need?"  all  20  operators  answered 
"No."  In  answer  to  "Were  there  other  areas  of  content  that  you  feel 
should  have  been  discussed  or  reviewed?"  16  operators  answered  "No." 
Topics  suggested  by  the  remaining  four  operators  were  (1)  physical- 
security  sensors  and  (2)  sensor- implantation  information. 

The  ninth  question  requested  the  operators  to  rate  each  component 
of  the  lessons  in  terms  of  clarity  of  presentation  and  difficulty  to 
answer.  Table  22  shows  the  results  of  the  operator  ratings.  The 
weighted  sums  were  computed  to  make  some  elementary  with in- lesson  com- 
parisons of  lesson  content,  practical  exercise  questions  (PEQ)  and 
criterion  exercise  questions  (CEQ)  as  well  as  across-lesson  compari- 
sons. The  weiqhted  sums  were  also  totaled  for  each  lesson  in  order 
to  assess  the  overall  effect  of  the  lesson. 

Note  that  the  lower  the  weighted  sum,  the  greater  the  clarity  as 
perceived  by  the  operators.  However,  the  lower  the  weighted  sum,  the 
greater  the  difficulty  as  perceived  by  the  operators.  Therefore,  a "1" 
on  the  rating  scale  means  very  clear  or  very  difficult.  Tlie  use  of 
these  data,  along  with  the  within-course  performance  data  and  the 
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Operator  Ratings  of  Clarity  and  Difficulty  on  Lesson  Materials 


Difficul 


Clarit; 


Not  Weiqhted  Very  Not  Weighted 

clear  sums  difficult  difficult  sums 


Two- sensor 
strings  & 
formula 
review 


3.  End-points  11  5 2 1 1 

PEQ  983 
CEQ  8 5 5 1 1 


4.  Column  length 

quantity  10  5 

PEQ  10  5 

CEQ  12  4 


Non- stairstep 
activation 
patterns  10  6 

PEQ  11  6 

CEQ  10  5 


6.  EMID 


7 .  MAGID 


8.  DIRID 


pretest  and  posttest  results,  would  allow  for  a more  effective  revision 
of  the  materials. 

The  following  summarizes  the  answers  to  "List  any  suggestions  you 
might  have  for  improving  a specific  lesson  or  component." 

• Have  this  kind  of  work  available  more  often; 

• Pick  up  all  materials  in  one  trip; 

• Make  note-taking  possible; 

• Use  better  equipment  (machines  in  training  course) ; 

• Give  10-minute  information  to  all  of  them; 

• Reword  some  of  the  lesson  sheets  to  increase  clarity; 

• Add  a little  more  information  to  all  of  them; 

• Break  all  sessions  down  to  about  3 or  4 hours  at  a time;  and 

• Read  lessons  aloud  and  explain  them. 


Job  Aid  Performance  Measures 


Six  target  variables  were  statistically  analyzed  to  determine  the 
effectiveness  of  the  nomograph  job  aid  in  combination  with  the  UGS  ruler 
TTi , T„,  target  type,  column  length,  target  quantity,  and  reporting 
time.  Table  23  presents  the  average  operator  performance  on  the  pre- 
test and  posttest  for  each  of  the  variables.  As  shown  in  the  table, 
all  of  the  variables  tested  are  statistically  significant.  It  is  con- 
cluded that  use  of  the  UGS  ruler  and  nomograph  substantially  improved 
operator  performance. 

The  UGS  ruler,  which  is  used  to  measure  TT^  and  TM  values,  resulted 
in  63%  and  121%  improvement,  respectively.  Note  that  the  operators  had 
previously  been  trained  and  given  UGS  rulers  (separate  from  the  nomo- 
graph) to  use  during  the  self-paced  training.  The  effect  of  this  on 
the  above-mentioned  comparisons  is  difficult  to  judqe.  However,  the 
improvement  scores  are  probably  lower  than  they  would  have  been  had 
the  operators  not  been  exposed  to  the  first  experiment.  Because  the 
operators  had  been  exposed  to  the  idea  of  accurately  measuring  TT^  and 
Tm  to  within  .1  minute  by  using  the  UGS  ruler,  they  were  inclined  to 
achieve  this  same  accuracy.  Prior  to  the  training  experiment  the 
operators  had  been  taught  to  round  off  their  time  estimations  to  the 
nearest  half-minute.  However,  over  half  of  them  were  estimatina  their 
time  values  to  the  .1  minute  during  the  above  pretest.  If  they  had 
retained  the  half-minute  criteria,  their  TT^  and  TM  correctness  values 
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(and  related  computations)  during  the  pretest  would  have  been  poorer 
than  they  actually  were.  Thus,  exposure  to  the  UGS  ruler  and  the  con- 
cept of  accurate  measurement  probably  reduced  the  pre-  and  posttest  dif- 
ferences (which  were  substantial) . 

In  summary,  the  following  percentage  improvements  occurred:  tar- 
get speed,  97%;  target  type,  7%;  column  length,  67%;  and  quantity,  41%. 
The  relatively  low  percentage  improvement  shown  for  target  type  should 
not  be  misunderstood.  Target  speed  is  the  major  determiner  of  target 
type,  and  the  percentage  improvement  for  target  speed  is  substantial. 

The  criterion  of  150  meters  per  minute  was  used  in  this  study,  i.e., 
targets  traveling  slower  than  this  speed  were  called  personnel  and  tar- 
gets traveling  equally  as  fast  or  faster  were  called  vehicles.  The 
number  of  times  in  which  increased  information  accuracy  (as  provided 
by  the  nomoqraph)  will  make  the  difference  between  calling  a particular 
target  personnel  or  vehicle  will  differ  depending  upon  how  close  the 
speed  is  to  150  meters  per  minute.  The  number  of  times  in  which  it 
oc  -urred  in  this  study  was  left  to  chance,  as  it  occurred  in  the  mate- 
rial obtained  from  the  field. 
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A 15%  improvement  over  pretest  time  occurred  in  time  taken  to  re- 
port on  each  target.  The  percentage  improvement  is  lower  than  had  been 
expected.  However,  analysis  of  what  the  job  aids  do  for  an  operator 
indicates  that  reporting  time  might  not  be  greatly  affected.  The  job 
aid  certainly  should  reduce  the  time  required  for  performing  calcula- 
tions. However,  measurement  of  TM  x TT^  with  the  ruler  should  require 
more  time  than  just  estimating  those  values . 

The  training  and  testing  of  the  utility  of  the  nomograph/UGS  ruler 
job  aid  combination  occurred  after  the  data  collection  on  the  individu- 
alized training  study.  Therefore,  the  operators  had  received  a lot  of 
practice  in  making  the  necessary  measurements  and  computing  the  speed 
and  quantity  of  targets.  However,  it  was  not  known  whether  or  not  the 
operators  had  reached  their  "normal"  level  of  competence.  If  they  had 
not,  then  the  additional  practice  received  during  Session  I could  in- 
crease their  scores  in  Session  II  but  could  appear  to  be  due  to  the 
use  of  the  nomograph/UGS  ruler. 


If  learning  or  practice  effects  occurred  in  Session  I (and  to  some 
extent  Session  II) , this  increase  should  be  evident  in  a comparison  of 
performance  on  the  problems  in  the  first  half  of  each  session  with 
that  on  the  problems  in  the  last  half  of  the  sessions.  This  assumes 
that  the  problems  in  the  first  half  are  equal  in  difficulty  to  those 
in  the  second  half.  Table  24  presents  these  data  for  the  speed  and 
quantity  computations.  Because  the  second  half  of  the  problems  in 
each  session  actually  resulted  in  lower  operator  performance  scores, 
it  is  concluded  that  no  learninq/practice  effects  occurred,  but  that 
the  differences  between  Session  I and  Session  II  were  due  to  the  use 
of  the  nomograph/UGS  ruler  job  aid. 


Table  24 


Number  Riqht  of  First  Half  Versus  Second  Half 
of  Test  on  Speed  and  Quantity  Estimations 


Session  I Session  II  Total 


1st  Half 

2d  Half 

1st  Half 

2d  Half 

1st  Half 

2d  Half 

Speed 

42 

34 

78 

64 

120 

98 

Quantity 

45 

43 

72 

68 

117 

111 

Practice  effects  apparently  occurred  on  the  time  variable.  Table 
25  shows  a marked  difference  between  the  first  and  second  halves  of 
both  Sessions  I and  II,  with  the  biqqest  difference  occurrinq  in  Ses- 
sion I.  A plot  of  the  individual  items  (Fiqure  3)  shows  this  effect 
more  clearly.  From  items  1 to  5 in  Session  I there  is  a decrease  in 
time,  after  which  it  tends  to  remain  constant.  The  averaqe  time  for 
the  first  five  items  was  siqnif icantly  hioher  (.05  level;  t = 2.34, 
df  = 8)  than  the  averaqe  for  the  first  five  items.  In  Session  II  there 
is  an  initial  "warmup"  or  perhaps  practice  on  the  first  or  first  four 
items  offered,  after  which  time/item  remains  fairly  constant.  There- 
fore, it  is  concluded  that  most,  if  not  all,  of  the  session  differ- 
ences found  in  the  time  variable  are  attributable  to  practice  (learninq, 
warmup,  etc.) . 


Table  25 

Average  Time  for  First  Half  of  Test 
Versus  Second  Half  of  Test 


Session  I 

Session  II 

Total 

1st  Half 

2d  Half 

1st  Half 

2d  Half 

1st  Half 

2d  Half 

Averaqe 

time 

22.1 

16.9 

18.5 

15.5 

40.6 

32.4 
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Figure  3.  Reporting  time  by  item. 

CONCLUSIONS  AND  RECOMMENDATIONS 

Conclusions 

1.  The  individualized  training  program  developed  from  an  analysis 
of  operator  errors  improved  the  performance  of  UGS  operators,  as 
follows: 

a.  An  18%  increase  in  the  number  of  detection  rights  (which  is 
one  target  per  operator  per  hour  on  the  average) . 

b.  About  half  of  the  above  increase  occurred  in  the  2-sensor 
string  condition. 

c.  The  largest  increase  in  detection  rights  occurred  in  the  hiqh- 
tarqet-activity  condition. 

d.  A 31%  increase  in  the  number  of  identification  rights  (which 
is,  on  the  average,  2-1/2  targets  per  operator  every  2 hours) . 
This  outcome  includes  the  increase  attributable  to  the  increase 
which  occurred  for  detection  rights. 
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e.  Most  of  the  increase  in  identification  rights  qiven  above 
occurred  in  the  high-activity  condition. 

f.  Session  X resulted  in  58%  detection  completeness  (riqhts 
divided  by  total  number  of  targets)  which  is  comparable  to 
the  55%  completeness  of  a previous  study  (Martinek,  Pilette, 
and  Biggs , 1974)  using  similar  (but  not  identical)  target 
scenarios  and  operators.  Session  II  performance  increased 
to  68%  as  a result  of  the  training. 

2.  No  significant  number  of  false  alarms  was  found. 

3.  If  a target  is  detected,  it  will  be  identified  right  (as  a vehicle 
or  personnel)  about  85%  of  the  time. 

4.  The  following  results  occurred  with  respect  to  additional  in- 
telligence information  associated  with  detectinc  and  reporting  targets : 

a.  A 63%  increase  in  the  number  of  correct  TT^  values; 

b.  A 58%  increase  in  the  number  of  correct  T values; 

M 

c.  A 64%  increase  in  the  number  of  correct  speed  values; 

d.  A 63%  increase  in  the  number  of  correct  quantity  values;  and 

e.  A 15%  reduction  in  the  time  for  target  reporting. 

5.  Answers  to  a questionnaire  qiven  to  the  operators  after  the 
training  indicated  a strong  favorable  reaction  to  the  self-paced, 
individualized  training  approach.  Thirteen  operators  indicated  that 
the  training  approach  was  "definitely  better"  than  the  conventional 
military  training  they  had  received.  The  remaining  seven  operators 
indicated  it  was  "somewhat  better."  None  of  the  operators  indicated 
it  was  "worse"  or  that  there  was  "no  difference"  between  the  two. 
Operator  reactions  were  obtained  on  all  aspects  of  the  training  as 
reported  in  the  Results  section.  Although  acceptance  of  the  UGS 
ruler  was  not  specifically  treated  in  the  questionnaire,  the  research- 
ers noted  that  such  acceptance  was  hiqh,  judqing  from  the  amount  of 
use  it  received  and  the  difficulty  encountered  in  getting  the  rulers 
back  from  the  operators. 

6.  In  the  second  study,  the  nomoqraph/UGS  ruler  job  aid  combination 
was  favorably  received  and  improved  operator  performance  on  the  fol- 
lowing variables. 

a.  63%  increase  in  correct  TT^  values; 

b.  121%  increase  in  correct  T values; 
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c.  97%  increase  in  correct  speed  values; 

d.  7%  increase  in  tarqet  identifications; 

e.  67%  increase  in  correct  length  of  column  values; 

f.  41%  increase  in  correct  quantity  values;  and 

g.  15%  reduction  in  reporting  time. 

7.  Use  of  3-  and  4-sensor  strings  produced  about  the  same  operator 
completeness  (80%  and  69%)  and  better  completeness  than  use  of  2-sensor 
strings  (48%) . However,  more  vehicle  targets  (presumably  easier)  oc- 
curred in  conjunction  with  the  3-  and  4-sensor  strings  than  with  the 
2-sensor  strings. 

8.  Considering  both  sessions,  a higher  completeness  is  found  during 
periods  of  low  target  activity  (76%)  than  high  target  activity  (65%)  . 

9.  After  training,  operator  performance  on  the  3-  and  4-sensor  strings 
was  77%  completeness  with  virtually  no  false  alarms . 

10.  Scenario  differences  were  found,  despite  the  attempt  to  eliminate 
them  as  a source  of  variation. 


Recommendat ions 

1.  The  improvements  in  operator  performance  that  occurred  as  a re- 
sult of  the  training  lead  to  the  recommendation  that  the  lesson  mate- 
rials (or  portions) , together  with  the  individualized  training  approach, 
be  refined  and  implemented  as  follows: 

a.  Provide  review  and  on-the-job  training  in  the  field,  and 

b.  Be  integrated  into  the  required  training  for  UGS  operators 
at  Fort  Huachuca,  Ariz. 

2.  It  is  further  recommended  that  the  nomograph  and  UGS  ruler  be 

a.  Taught  during  required  traininq  for  UGS  operators  at  Fort 
Huachuca,  Ariz.,  and 

b.  Included  as  standard  issue  with  the  R0376,  the  BASS  III,  and 
any  newly  developed  recorders. 

3.  An  additional  study,  controlling  for  target  differences,  should 
be  done  comparing  the  2-,  3-,  and  4-sensor  string  conditions.  If 
operator  performance  is  the  only  criterion,  this  study  indicates  that 
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three  sensors  produce  as  much  target  information  as  four.  Conceivably, 
two  sensors  may  be  as  good  as  three  or  four. 

4.  A report  of  the  detection  of  a target  should  be  sent  forward 
using  two  sensors  (or  more  in  cases  where  two  are  not  sufficient) 
with  a followup  report  giving  speed,  number  of  objects,  and  type 

of  target.  This  will  provide  an  alert  function  at  the  earliest  pos- 
sible time. 

5.  Operator  performance  is  at  an  acceptable  level. 

6.  In  future  tests  and  research  concernina  UGS,  scenario  differences 
probably  will  occur  and  should  be  controlled  experimentally  and/or 
statistically. 
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Appendix  A 


LESSON— TWO-SENSOR  STRINGS  AND  FORMULA  REVIEW 


To  enhance  awareness  in  detecting  2-sensor  strings  and  to  provide 
recall  or  basic  training  in  computing  the  column- length  formula. 


Detecting  target  patterns  from  2-sensor  strings  are  important  be' 
cause  they  can  frequently  provide  as  much  information  as  strings  with 
more  sensors. 


Two-sensor  strings  may  be  deployed  for  the  following  reasons 


1.  Along  roadways  when  there  is  a shortage  of  sensors 


2.  On  mountain  or  hilltop  areas  to  detect  movement,  and 


3.  Where  only  two  sensors  are  functioning  out  of  a larger  string 
because  of  failure  or  damage. 


When  2-sensor  strings  are  deployed  together  in  the  same  sensor 
field  with  3,  4,  and  5-sensor  strings,  it  is  easy  to  miss  their  ac- 
tivation patterns.  Two-sensor  string  patterns  can  easily  be  over- 
looked because  the  larger  string  patterns  are  much  more  obvious . The 
2-sensor  string  patterns  may  yield  equally  relevant  information  con- 
cerning enemy  movements,  but  to  detect  them  sometimes  requires  more 
careful  observation. 


In  order  to  maximize  the  chances  of  detecting  2-sensor  strings 
the  following  guidelines  are  helpful: 


Check  the  Sensor  Data  Record  and  Sketch  Maps  carefully  for 
the  location  and  pen  numbers  of  2-sensor  strings. 


When  activations  occur  on  both  pens  in  the  familiar  stair 
step  pattern,  the  chances  are  good  it  is  a target. 


When  activations  occur  on  both  pens,  but  not  in  the  familiar 
stairstep  pattern,  they  may  or  may  not  represent  a target. 
Check  other  sensors  in  the  area  to  determine  if  the  activa- 
tions could  be  the  result  of  noise  activity  such  as  artillery 
or  aircraft. 


The  Column- Length  Formula  requires  a minimum  of  2-sensors;  there- 
fore, it  can  be  used  on  the  2-sensor  strings. 

To  review,  the  Column-Length  Formula  is 

LC  ■ ^ <TV  - CDR- 

where 

Lc  = length  of  the  tarqet  column  (in  meters) , 

D = distance  between  the  sensors  (in  meters, 

Tm  = time  difference  (in  minutes)  between  the  mid-point 
of  the  first  sensor  activation  pattern  and  the  mid- 
point of  the  second  sensor  activation  pattern, 

TT^  = total  time  (in  minutes)  of  the  first  sensor  activation 
pattern , and 

CDR  = Combined  Detection  Range  (in  meters) . The  sum  of  the 
detection  range  of  the  first  sensor  and  the  detection 
range  of  the  second  sensor.  CDR  is  used  instead  of  DR 
to  eliminate  confusion  between  the  detection  range  (DR) 
of  one  sensor  and  the  detection  range  of  two  sensors 
when  they  are  added  together  (CDR) . 

The  following  values  were  measured  from  the  2-sensor  string  in 
Figure  1 (pens  3,4)  . Check  them  for  correctness. 

T = 2 min. 

M 

TT^  = 3 min. 

The  Formula  is  solved  using  the  following  assumptions: 

D = 500  meters  (m) 

CDR  = 600  meters  (m) 

Lc  = §"  (TT^  - CDR 
m 

= 500m  (3  min>)  _ 600m 

2 min. 

= 250  — (3  min.)-  600m 
min . 

= 750m  - 600m 

L = 150m 
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Since  the  speed  of  the  target  is  250  m/min,  it  is  considered 
vehicular.  To  determine  the  number  (quantity)  of  targets  in  the 


column  the  L is  divided  by  50m  as  shown, 
c 


Quantity  (Q)  - * 3 vehicles. 

50m  50m 


Of  course,  if  the  speed  is  less  than  150  m/min  the  L would  be 
divided  by  5m. 


WHEN  YOU  ARE  FINISHED  WITH  THIS  LESSON  TAKE  IT  TO  THE  TRAINING 
MONITOR  (STATION  2)  AND  PICK  UP  YOUR  PRACTICAL  EXERCISE  QUESTION  SHEET. 


........ ...... ... 
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Appendix  B 

SUBJECT  INSTRUCTIONS 
A.  Test  Procedure  Training 
MONITOR:  Read  the  following  instructions : 

As  you  already  know,  you  have  been  selected  to  participate  in  a 
research  effort  to  study  UGS  performance  under  various  conditions  as 
they  might  exist  in  the  field.  We  have  been  asked  by  the  Department 
of  the  Army  to  administer  this  exercise  to  you,  collect  performance 
data,  analyze  the  results,  and  submit  a report.  I want  to  make  it 
clear  at  the  outset  that  our  objective  is  not  to  isolate  each  of  you 
and  attach  a proficiency  score  to  each  of  you.  Our  objective  is  to 
determine  the  total  capability  of  the  UGS  information  potential  as  it 
combines  both  the  person  (you)  and  the  machine  output  (the  UGS  record) . 
However,  your  performance  will  be  scored  and  given  to  your  platoon 
leader  so  that  he  will  have  some  idea  about  how  good  you  are  compared 
to  the  rest  of  the  group  in  the  readout  of  the  R0376  event  recorder. 

Since  other  groups  of  UGS  operators  have  taken  or  will  be  taking 
this  exercise,  I will  be  looking  frequently  at  my  notes  to  be  sure  ev- 
eryone gets  the  same  information.  You  will  have  opportunities  through- 
out this  entire  program  to  ask  questions. 

To  summarize  the  entire  exercise,  you  will  be  given  UGS  sensor 
records  (X-T  plots)  to  interpret.  These  plots,  for  the  most  part,  are 
target  activations  that  have  been  collected  under  various  field  exer- 
cises primarily  in  the  Fort  Hood  area.  Involved  in  these  exercises 
were  varying  numbers  of  tanks.  Armored  Personnel  Carriers  (APC's),  and 
personnel.  These  tests  also  contain  noise  activations  which  are  typical 
of  wartime  operation  such  as:  fixed-wing  and  helicopter  activity, 
malfunctioning  and  unreliable  sensors,  radio  interference,  weather/ 
wind  activity,  and  artillery  shell  bursts.  All  we  ask  is  that  you 
interpret  the  X-T  plots  to  the  best  of  your  ability  and  try  to  make 
sense  out  of  what  sometimes  might  appear  to  you  to  be  rather  difficult. 
Let  me  stress  that  we  have  tried  to  make  these  records  as  realistic 
as  we  could. 

You  should  interpret  these  X-T  plots  as  though  you  are  under  bat- 
tlefield conditions.  However,  in  wartime  it  will  be  a lot  worse.  You 
won't  slow  down  then,  because  your  life  may  depend  on  how  well  you  do 
your  job.  However,  at  the  risk  of  sounding  melodramatic,  keep  in  mind 
that  you  should  try,  because  there  is  a great  need  for  the  Army  to 
find  out  how  typical  UGS  specialists  will  be  able  to  function  under 
battle  conditions  that  we  selected. 

You  are  important  because  you,  as  a group,  represent  the  hundreds 
of  specialists  that  have  graduated  and  will  graduate  from  the  UGS  school 
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for  a long  time  to  come.  Army  deployment  plans  for  UGS  equipment  and 
personnel  will  be  partly  influenced  based  upon  what  you  can  do. 


MONITOR:  Pass  out  the  packets  and  read  the  following  instructions: 

The  next  sheet  should  be  a TARGET  LOG.  On  this  sheet  you  will 
record  most  of  the  target  information  you  would  regularly  record  on  a 
SENSOR  ACTIVATION  SPOT  REPORT  plus  additional  information.  Note  the 
first  column  of  the  target  log.  This  is  the  target  pattern  number. 

For  activation  patterns  on  the  chart  paper  that  you  think  are  targets, 
circle  the  target  on  the  chart  paper  and  number  it.  Record  the  number 
in  the  first  column.  For  each  target  that  you  detect,  always  number 
it  and  record  it  on  your  target  log.  If  you  detect  a target,  but 
later  feel  it  is  not  a target,  do  not  erase  what  you  have  written, 
but  simply  write  "No  Target"  and  start  recording  the  next  target  on 
the  space  below.  Number  all  your  targets  consecutively  in  sequence. 

In  column  2 record  the  clock  time  that  you  made  a target  detec- 
tion and  round  it  off  to  the  nearest  minute.  You  may  use  your  wrist 
watch,  but  be  sure  it  is  synchronized  with  the  wall  clock. 

In  column  3 record  the  pen  numbers  of  the  sensor  string . 

In  column  4 record  how  confident  you  are  that  what  you  think  is  a 
target  really  is  a target.  Use  the  terms  for  your  confidence  that  are 
shown  at  the  bottom  of  the  target  log  sheet:  positive,  high,  50/50,  and 
low.  Tell  us  how  confident  you  are  that  the  activation  pattern  you  are 
looking  at  really  is  a target,  assuming  that  you  will  be  sending  this 
information  to  your  Company  Commander  during  a battle  situation.  He 
knows  you  are  doing  your  best  but  he  wants  to  know  how  confident  you 
are  in  the  information  you  are  sending  to  him.  If  you  are  not  confi- 
dent in  your  information  then  he  will  place  a higher  weight  on  the 
information  being  supplied  to  him  by  other  intelligence  sources  such 
as  realtime  IR  reconnaissance,  radar  or  electronic  intelligence.  The 
confidence  estimate  should  relate  only  to  your  detection  of  a target 
and  not  necessarily  to  any  of  the  following  information. 

In  column  5 record  the  direction  of  movement  of  the  target.  Use 
the  terms  NW,  NE , SW,  and  SE. 

In  column  6 compute  the  speed  of  the  target  using  the  formula 

Speed  (S)  = D/Tm 

where  D = distance  between  sensors,  and 

T = time  of  activations  from  center  of  the  first  sensor 
M activation  to  the  center  of  the  second  sensor  pattern. 

Show  your  calculations  for  speed  in  the  space  provided. 

Round  off  your  speed  to  the  nearest  meters /minutes . 
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In  column  7 record  the  target  type,  whether  personnel  or  vehicu- 
lar. If  the  speed  is  less  than  150  meters/minute  call  the  target 
"personnel"  and  write  P.  If  the  target  speed  is  greater  than  150 
meters/minute  call  the  target  "vehicles"  and  write  V. 

In  column  8 compute  the  length  of  the  target  column  using  the 
formula 

L = S (TT^  - Dr 

where  S « speed  calculated  and  recorded  in  column  6, 

TT^  = total  time  the  first  sensor  was  activated,  and 

Dr  = detection  range  which  is  the  detection  radius  of  the 
first  sensor  plus  the  detection  radius  of  the  second 
sensor  added  together.  In  this  test  the  detection 
radius  of  all  sensors  in  a string  is  the  same. 

Substitute  the  values  in  the  formula  in  the  spaces  provided  and  show 
your  work.  Do  your  actual  multiplication  on  scratch  paper.  Round  off 
your  numbers  where  appropriate. 

In  column  9 record  the  quantity  or  number  of  units  in  the  target. 
If  the  target  is  personnel  divide  the  length  of  the  column  by  5 meters 
to  determine  the  number  of  personnel.  If  the  target  is  vehicles  divide 
the  length  of  the  column  by  50  meters  to  determine  the  number  of 
vehicles. 

In  column  10  record  the  clock  time  in  which  you  are  finished  with 
the  target  and  are  ready  to  send  the  information  to  your  Company 
Commander . 

The  sequence  of  the  information  required  on  the  target  log  was 
planned,  and  we  want  you  to  try  to  use  this  sequence.  Does  anyone 
have  any  reservations  about  this  sequence? 

MONITOR:  Discuss  the  implant  sketch  (Field  I)  and  read  the  following: 

Remove  your  target  log  and  place  it  aside.  The  next  sheet  should 
be  an  IMPLANT  SKETCH,  which  is  marked  on  the  upper  right-hand  side — 
Pretest  Part  I.  As  you  know,  implant  sketches  show  the  location  of 
the  sensor  strings. 

During  the  next  couple  of  days,  you  will  be  using  implant  sketches 
very  similar  to  this  one.  The  implant  sketches  will  show  the  location 
of  the  sensors  along  the  roads  and  trails.  Give  the  pen  number  for 
each  sensor,  the  string  number,  and  the  distance  between  the  sensors. 
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As  you  can  see,  the  spatial  relationships  between  the  sensors  are 
not  drawn  to  scale.  This  is  the  case  with  all  the  implant  sketches 
you  will  be  qiven  during  this  entire  exercise.  The  purpose  of  this 
implant  sketch  is  merely  to  present  you  with  the  information  that  you 
will  need.  There  will  also  be  times  in  which  you  may  disagree  as  to 
the  placement  of  the  sensor  strings  in  the  various  deployment  patterns. 
Although  certain  string  deployments  may  differ  from  what  you  feel  or 
have  been  taught  is  correct,  accept  the  deployments  presented,  keepinq 
in  mind  that  in  operational  situations  there  are  many  factors  involved 
in  deploying  sensors.  The  sensor-string  deployments  are  the  ones  used 
in  the  Fort  Hood  tests  and  are  adequate  for  purposes  of  this  exercise. 

The  implant  sketches  will  also  provide  you  with  information  as  to 
whether  the  string  contains  a MAGID,  EMID,  or  DIRID  sensor.  In  this 
implant  sketch  (Pretest — Field  I)  there  is  one  MAGID. 

Place  the  implant  sketch  aside.  The  next  sheet  should  be  a SENSOR 
PROGRAM  RECORD.  This  sheet  provides  the  string  number  and  field  desig- 
nation, sensor  type,  and  detection  ranges  of  the  various  sensors.  This 
sheet  contains  four  sections  to  provide  this  information  for  four  dif- 
ferent sensor  fields.  On  this  sheet  three  sensor  fields  are  shown: 

I,  II,  and  III.  Right  now  we  will  be  concerned  with  only  the  first 
sensor  field  labeled  Roman  Numeral  I.  The  sensor  record  or  read-out 
sheets  you  probably  used  to  work  with  contain  a great  deal  more  informa- 
tion including  geographical  coordinates,  sensor  ID,  sensor  frequency, 
mode,  and  inhibit  times.  The  ones  we  will  work  with  are  simplified  for 
purposes  of  this  exercise. 

Notice  that  for  detection  radius,  a value  is  given  for  vehicles 
and  for  personnel.  Detection  radius  varies  greatly  for  vehicles  and 
personnel  depending  upon  the  sensor  type,  soil  type,  and  gain  setting. 


MONITOR:  Administer  Pretest  Part  I and  read  the  following: 

Place  your  target  log,  implant  sketch,  and  sensor  record  in  a 
convenient  place  near  your  event  processor  which  is  already  loaded 
with  our  X-T  chart  paper.  Place  your  packet  aside.  We  will  work 
through  a 30-minute  practical  exercise,  in  which  we  will  use  the  mate- 
rials we  have  just  discussed.  Fill  out  your  target  log  and  use  your 
implant  sketch  and  sensor  record.  As  you  detect  patterns  that  you 
think  are  targets,  circle  the  pattern  on  the  chart  paper  with  your 
pencil  and  place  the  number  of  the  target  next  to  the  circle.  Then 
fill  out  your  target  log  by  first  recording  this  number  in  column  1. 
Are  you  ready?  Start  monitoring. 


MONITOR:  Turn  on  drive  mechanisms  and  check  each  subject  to  be  sure 
all  are  working  with  the  proper  materials.  When  the  first  group  of 
noise  activations  has  completely  appeared,  find  out  if  anyone  called 
them  targets.  When  the  first,  second,  and  third  targets  have  each 


appeared  (in  different  time  frames) , find  out  who  did  not  detect  them 
as  targets,  and  assist  these  people.  For  the  first  target,  work  out 
the  calculations  on  the  blackboard  if  necessary.  Find  out  how  this 
class  is  performing  and  give  assistance  individually  or  for  the  entire 
class  as  required.  Emphasize  aspects  of  the  procedure  that  are  not 
being  performed.  Check  each  man  to  be  sure  he  is  properly  filling  in 
all  the  information  blanks  on  the  target  log. 

At  the  end  of  the  30-minute  period,  turn  off  the  event  processors 
no  more  than  1/2-inch  above  the  line  separating  the  30-minute  periods 
and  review  procedures  where  appropriate. 


MONITOR:  Read  the  following: 

During  the  practical  exercises  you  will  be  given  today,  you  will 
be  monitoring  various  30-minute  chart  preparations.  After  each  30- 
minute  period  you  will  be  given  a new  implant  sketch.  We  will  begin 
the  next  30-minute  period  shortly.  Get  your  packets  and  take  out  the 
implant  sketch  marked  Pretest--Part  II  in  the  upper  right-hand  corner. 
Do  you  have  any  questions  concerning  this  one?  Remember,  you  are  num- 
bering your  targets  consecutively  (in  sequence)  so  if  you  need  addi- 
tional target  logs,  let  us  know  and  we  will  supply  them.  Be  sure  you 
are  working  with  the  appropriate  Sensor  Record.  For  the  next  30-minute 
segment  you  will  be  working  with  the  Sensor  Record,  labeled  Pretest  II. 
Are  there  any  questions  before  we  begin  the  next  30-minute  seqment? 

All  right,  we  will  now  start. 


MONITOR:  Conduct  Pretest — Part  II 

Conduct  in  a similar  manner  as  Part  I except  with  somewhat  less 
involvement  of  the  monitors  during  the  test.  Check  what  procedural 
problems  are  still  occurring.  At  the  end  of  the  30  minutes,  turn  off 
the  equipment.  Address  the  entire  class  on  procedural  problems  and 
reiterate  where  necessary. 


B.  Pretest  and  Posttest  Training 
MONITOR:  Read  the  following: 

You  will  now  be  given  four  30-minute  segments.  You  won’t  be 
stopped  after  each  one,  but  you  will  proceed  through  all  four  segments 
as  though  you  were  working  a 2-hour  shift  in  the  field.  Take  the  mate- 
rial out  of  your  packets.  The  first  two  sheets  will  be  the  Sensor 
Record  Sheet  and  Implant  Sketch  for  your  first  30-minute  period,  and 
as  you  can  see,  they  are  the  same  as  those  you  have  used  previously. 

I would  like  to  remind  you  that  the  wall  clock  is  right  there 
(Monitor:  point  to  it)  to  get  the  times  required  on  the  target  log. 
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If  you  are  going  to  use  your  wristwatch,  set  it  with  the  wall  clock 
now.  We  will  not  be  assisting  you  in  any  way  to  aid  you  in  your  tar- 
get analysis.  We  will,  however,  be  assisting  you  to  make  sure  you 
are  looking  at  the  right  sensor  record  and  implant  sketch  throughout 
the  exercise.  If  you  have  any  questions  with  regard  to  whether  or  not 
you  are  using  the  proper  sensor  record  or  implant  sketch,  please  do 
not  hesitate  to  ask.  Be  sure  to  circle  the  activation  patterns  that 
you  think  are  targets.  Number  these  patterns  consecutively  and  record 
in  the  first  column  of  the  target  log.  Are  you  ready?  Begin. 


MONITOR:  Turn  the  equipment  on.  Check  to  see  that  all  the  equipment 
is  running  satisfactorily.  Every  30  minutes  the  monitor  will  make 
sure  that  each  subject  is  using  the  proper  Implant  Sketch  and  Sensor 
Record . 


C.  Individualized  Training  Administration 


MONITOR:  Read  the  following  instructions: 

Yesterday  you  worked  on  a practical  exercise  that  required  you  to 
read-out  an  X-T  plot.  We  haven't  scored  it  yet,  but  from  past  expe- 
rience with  UGS  operators  such  as  yourself,  we  can  say  that  ill  of  you 
made  errors.  Today's  training  session  will  be  directed  toward  errors 
previously  made  by  other  UGS  operators.  We  assume  that  you  also  made 
similar  errors.  We  believe  this  training  plus  the  practice  you  re- 
ceived will  increase  your  effectiveness  as  an  UGS  operator.  It  also 
keeps  you  out  of  PT  and  clean-up  details. 


This  type  of  training  has  been  used  successfully  in  other  educa- 
tional systems  and  has  been  approved  by  training  specialists.  How 
well  it  works  out  here  is  one  of  the  questions  we  are  trying  to  answer. 
You  will  be  asked  to  assess  it,  afterwards,  in  a questionnaire  to  get 
the  UGS  operator's  point  of  view. 


For  most  of  you  this  traininq  will  be  quite  different  from  any 
training  you  have  received  in  the  past.  The  training  system  has  been 
designed  to  allow  you  to  proceed  at  your  own  pace,  test  your  own  knowl- 
edge after  short  seqments  of  instruction,  obtain  immediate  feedback 
on  your  mastery  of  the  concepts,  and  receive  expert  assistance  when 
you  need  it  and  want  it.  Those  who  are  familiar  with  the  concepts 
can  proceed  through  the  lessons  at  a fast  pace,  while  those  who  need 
assistance  can  receive  it  when  they  need  it. 

In  this  self-paced  training  system  the  content  is  organized  into 
distinct  lessons,  each  covering  a short  segment  of  instruction.  Each 
lesson  contains  a Lesson  Sheet,  a Practical  Exercise  Question  Sheet, 
a Practical  Exercise  Answer  Sheet,  and  a Criterion  Exercise  Question 
Sheet. 
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1.  The  Lesson  Sheet  is  divided  into  three  sections:  Objective, 
Purpose,  and  Concept.  The  Objective  section  states  what  you 
should  be  able  to  do  at  the  end  of  the  lesson;  the  Purpose 
section  discusses  the  importance  of  the  lesson,  and  the 
Concept  section  contains  the  instructional  materials  with 
examples . 

2.  The  Practical  Exercise  Question  Sheet  contains  questions  on 
the  material  discussed  on  the  Lesson  Sheets.  These  questions 
are  intended  to  allow  you  to  test  your  mastery  of  the  concepts 
in  the  lesson. 

3.  The  Practical  Exercise  Answer  Sheet  provides  you  with  the 
correct  answers  to  the  questions  and  will  allow  you  to  check 
your  own  knowledge. 

4.  The  Criterion  Exercise  Sheet  contains  questions  relating  to 
information  on  the  Lesson  Sheet.  These  questions  test  the 
same  thing  as  the  practical  exercise  questions  and  are  the 
final  check  for  mastery  before  going  on  to  the  next  lesson. 

For  this  exercise  there  are  two  monitors  and  one  subject  matter 
expert.  They  will  assist  you  and  monitor  your  progress  through  the 
system. 

1.  One  monitor  will  be  located  at  Station  1 and  will  control 
handout  of  the  Lesson  Sheet  and  he  will  score  your  CRITERION 
EXERCISES . 

2.  The  second  monitor  will  be  located  at  Station  2 and  will 
control  all  activities  having  to  do  with  the  PRACTICAL 
EXERCISES. 

3.  The  subject  matter  expert  will  be  located  at  Station  3 and 
will  provide  content  assistance  whenever  it  is  needed. 

There  are  five  basic  steps  to  be  completed  for  each  lesson:  Read 
the  Lesson  Sheet,  work  the  Practical  Exercise  Questions  (PEQ's),  check 
your  PEQ's,  answer  the  Criterion  Exercise  Questions  (CEQ's) , and  have 
the  Test  Monitor  check  the  CEQ's. 

The  specific  steps  for  each  lesson  are  as  follows: 

1.  Get  the  Lesson  Sheet  from  Station  1,  return  to  your  desk, 
and  study. 

2.  Take  the  Lesson  Sheet  to  Station  2 and  qive  it  to  the  Test 
Monitor. 


a.  Pick  up  the  Practical  Exercise  Questions  (PEQ’s). 

b.  Return  to  desk  and  answer  the  PEQ's — answer  all  questions 
on  the  sheet . 

3.  Take  completed  PEQ  sheet  back  to  Station  2. 

a.  Show  the  Test  Monitor  that  you  have  answered  the  questions. 

b.  Pick  up  PEQ  Answer  Sheet  and  Lesson  Sheet. 

c.  Return  to  desk,  check  your  answers,  and  resolve  all  incor- 
rect responses.  Do  not  erase  or  scratch  out  any  of  your 
original  answers.  We  need  to  know  your  first  responr.es 

so  that  we  can  improve  the  materials.  If  you  need  assis- 
tance, qo  to  Station  3. 

4.  When  you  feel  you  understand  all  the  material . take  the  Lesson 

Sheet,  the  PEQ's,  and  PEQ  Answer  Sheet  back  r.o  Station  2. 

5.  Take  completed  CEQ's  to  Station  1 (Note:  Station  1 is  where 

you  picked  up  the  Lesson  Sheet  at  the  very  beqiminq)  ; the 

Test  Monitor  will  check  your  answers. 

a.  If  you  got  them  all  correct,  pick  up  the  next  Lesson  Sheet 
from  the  same  Test  Monitor  and  complete  it  as  ^ou  did  this 
one. 

b.  If  you  missed  any  questions,  return  to  Station  2,  and  aet 
all  your  materials  for  the  i'sson.  Return  to  your  desk 
and  re-answer  the  questions  y>~'.»  missed.  Do  not  erase 
your  oriqinal  answer.  We  need  that  information  in  order 

to  improve  the  materials.  Write  the  number  of  the  question 
you  missed  on  the  back  of  the  CEQ  Sheet  and  put  your  sec- 
ond answer  there.  Take  your  CEQ’s  back  to  Station  1 to  be 
checked.  If  you  missed  them  aqain,  you  will  be  required 
to  go  to  Station  3 for  assistance.  (Take  all  your  mate- 
rials with  you.)  If  you  got  them  correct,  you  may  proceed 
with  the  next  lesson. 

Are  there  any  questions  you  may  have  about  the  materials,  personnel, 
and/or  procedures?  Go  to  Station  1 and  begin. 
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ASSESSMENT  OF  PRACTICE  EFFECTS  IN  INDIVIDUALIZED  TRAINING 

Because  of  the  shortage  of  operators  to  participate  in  the  indi- 
vidualized training  study,  it  was  impossible  to  run  a control  group 
to  determine  the  effect  of  practice.  Although  seme  practice  obviously 
occurred,  it  is  possible  to  show  that  its  effect  on  detection  and 
identification  performance  is  negligible.  Table  C-l  shows  detection 
and  identification  rights  as  a function  of  30-minute  scenario  segments. 
Operator  performance  for  detection  rights  and  identification  rights  is 
presented  across  the  30-minute  segments.  Segments  1,  2,  5,  and  6 are 
low- target-activity  segments,  and  segments  3,  4,  7,  and  8 are  high- 
activity  segments. 


Table  C-l 

Detection  and  Identification  Rights 


Session  I 


Session  II 


30-minute  segments 
in  order  administered 


30-minute  segments 
in  order  administered 


Total 

number 

of 


1 

2 

3 

4 

5 

6 

7 

8 

rights 

Detection  rights 

Odd  segments 
Even  segments 

51 

31 

58 

68 

52 

39 

75 

80 

236 

218 

Identification 

rights 

Odd  segments 
Even  segments 

43 

27 

46 

50 

50 

35 

64 

68 

203 

180 

For  learning  or  practice  effects  to  have  taken  place,  the  total 
number  of  rights  for  the  even-numbered  segments  would  have  to  be 
greater  than  the  total  number  for  the  odd-numbered  segments  (assuming 
equal  target  difficulty  over  all  segments) . However,  for  both  detec- 
tion and  identification  rights,  the  total  number  of  rights  for  the 
even-numbered  segments  is  below  that  for  the  odd-numbered  segments. 
These  results  strongly  support  the  contention  that  practice  or  learning 
effects  did  not  have  an  important  impact  on  the  operator  performance 
increases  and  that,  therefore,  these  increases  are  due  to  the  training. 


In  addition,  previous  research  (Martinek,  Piletfce,  and  Biggs, 
1974)  has  shown  no  increases  in  performances  associated  with  8 hours 
of  practice  on  scenarios  highly  similar  to  those  used  in  this  study. 
Moreover,  the  operators  used  in  that  study  had  just  graduated  from 
school  and,  therefore,  should  show  the  largest  increases  attributable 
to  practice  (i.e.,  experience). 


DISTRIBUTION 


ARI  Distribution  List 


4 OASO  IMRRAI 
2 HQOA  (DAMI  CSZ) 

1 HQOA  (DAPEPBR 
1 HQOA  (DAMAAR) 

1 HQOA  (DARE  HRE  PO) 
t HQOA  (SGRD  101 
1 HQOA  I DAMI  DOT  C> 

1 HQDA  (DAPC  PMZ  A) 

1 HQDA  IOACHPPZ  A) 
t HQOA  (DAPE  HREI 
t HQOA  (OAPE  MPO  CI 
1 HQOA  (OAPE  DW) 

1 HQDA  (DAPE-HRl) 

1 HQDA  (DAPE  CPSt 
1 HQOA  (DAFD  MFA) 

I HQOA  IDAHO  ARS  P) 

1 HQDA  (DAPC  PAS  A) 
t HQOA  IOUSA-OR) 

1 HQOA  (OAMO  RQR) 

1 HQOA  IDASG) 
t HQOA  (OAIOPII 

1 Chi.(.  Consult  Di«  IDA  OTSGI.  Adelphi,  MO 
1 Mil  Ant.  Hum  Rat.  ODORRE.  OAO  (ERLS) 
t HOUSARAL,  APOSaattla,  ATTN:  ARAGP-R 
1 HQ  Flat  Army.  ATTN:  AFKA-OI-TI 
Z HQ  Flftti  Army,  Ft  Sam  Houston 
1 Dir.  Army  St*  StudlatOtc,  ATTN:  OAVCSA  (DSP) 

1 Ofc  Chief  ol  Stf,  Studies  Otc 
1 OCSPER.  ATTN  CPS/OCP 
1 The  Army  Lib.  Pentagon.  ATTN:  RSB  Chiat 
I Tha  Army  Lib.  Pentagon,  ATTN:  ANRAL 
1 Otc.  Ant  Sact  ot  tba  Army  (RRD) 

1 Tach  Support  Ofc,  OJCS 

1 USASA,  Arlington.  ATTN:  IARD  T 

f USA  Rich  Ofc.  Durham.  ATTN;  Ufa  Sclancat  Olr 

2 USARIEM.  Natick.  ATTN.  SGRD-UE  CA 

1 USATTC,  Ft  Clayton.  ATTN:  STETC  MQ-A 
t USAIMA,  Ft  Bragg.  ATTN:  ATSUCTDOM 
t USAIMA.  Ft  Bragg.  ATTN:  Merquat  Lib 
t US  WAC  Ctr  R Seh,  Ft  McClallan.  ATTN:  Lib 
I US  WAC  Ctr  R Sch.  Ft  McClallan.  ATTN:  Tng  Olr 
I USA  Quartarmattar  Sch.  Ft  Laa.  ATTN:  ATSM  TE 
1 Intalliganca  Malarial  Day  Ofc.  EWL.  Ft  Holablrd 
1 USA  SE  Signal  Sch.  Ft  Gordon.  ATTN:  ATSO  EA 
1 USA  Chaplain  Ctr  R Sch,  Ft  Hamilton,  ATTN:  ATSC  TE  RD 
t USATSCH,  Ft  Euttlt,  ATTN:  Educ  Advlaor 

1 USA  War  Collage.  Carlisle  Barrack.,  ATTN:  Lib 

2 WRAIR,  Neuropsychiatry  Olv 
t DLL  SOA.  Monterey 

I USA  Concept  Anal  Agey.  Bathasda.  ATTN:  MOCAWGC 
1 USA  Concept  Anal  Agcy,  Bathasda.  ATTN:  MOCA  MR 
1 USA  Concept  Anal  Agcy.  Bathasda.  ATTN:  MOCAJF 
t USA  Artie  Test  Ctr.  APO  Saattla,  ATTN:  STEAC  MO  ASI 
1 USA  Artie  Test  Ctr.  APOSaattla,  ATTN:  AMSTE  PL  TS 
1 USA  Armament  Cmd,  Redstone  Arsenal.  ATTN:  ATSK  TEM 
1 USA  Armamant  Cmd,  Rock  Island.  ATTN:  AMSAR-TOC 
1 FAANAFEC.  Atlantic  City.  ATTN:  Library 
f FAA  NAFEC.  Atlantic  City,  ATTN:  Hum  Engr  Br 

1 FAA  Aeronautical  Ctr.  Oklahoma  City,  ATTN:  AAC  44D 

2 USA  Fid  Arty  Sch,  Ft  Sill.  ATTN  Library 
) USA  Armor  Sch,  Ft  Knox.  ATTN:  Library 

1 USA  Armor  Sch.  Ft  Knox,  ATTN:  ATSB  DI  E 
t USA  Armor  Seh,  Ft  Knox.  ATTN:  ATSB  DT  TP 
I USA  Armor  Seh.  Ft  Knox,  ATTN:  ATSB  CO  AO 


2 HCUSACDEC.  Ft  Or d ATTN:  Library 
. HOUSACOEC.  Ft  Ord.  ATTN;  ATEC-EX-E-Hum  Factors 
2 USAEEC.  Ft  Baniamln  Harrison.  ATTN:  Library 
1 USAPACOC.  Ft  Beniamin  Harrison.  ATTN:  ATCP-HR 
I USA  Comm -Elect  Sch.  Ft  Monmouth.  ATTN:  ATSN-EA 
I USAEC.  Ft  Monmouth,  ATTN:  AMSEL-CT -HOP 
I USAEC.  Ft  Monmouth.  ATTN:  AMSEL-PA-P 
1 USAEC.  Ft  Monmouth.  ATTN:  AMSEL-SI-CB 
1 USAEC.  Ft  Monmouth.  ATTN:  C.  Fad  Das  Br 
1 USA  Materials  Sys  Anal  Agcy.  Aberdeen.  ATTN:  AMXSV-P 
f Edgewood  Arsenal,  Aberdeen,  ATTN:  SAREA-BL-H 

1 USA  Ord  Ctr  R Sch.  Aberdeen.  ATTN:  ATSL-TEM-C 

2 USA  Hum  Engr  Lab,  Aberdeen,  ATTN:  Library/DIr 

1 USA  Combat  Arms  Tng  Bd,  Ft  Banning,  ATTN:  Ad  Supervisor 
I USA  Infantry  Hum  Rich  Unit.  Ft  Banning.  ATTN:  Chief 
1 USA  Infantry  Bd,  Ft  Banning,  ATTN:  STEBC-TE-T 
1 USASMA,  Ft  Bliaa,  ATTN:  ATSS  -LRC 
I USA  Air  Oaf  Sch.  Ft  Bliss.  ATTN.  ATSA  CTO-  ME 
1 USA  Air  Oaf  Sch.  Ft  Bliu.  ATTN:  Tach  Lib 
I USA  Air  Oaf  Bd.  Ft  Bliu.  ATTN:  FILES 
1 USA  Air  Oaf  Bd.  Ft  Blits.  ATTN:  STEBD-PO 
t USA  Cmd  R General  Stf  Collage,  Ft  Leavenworth,  ATTN:  Lib 
f USA  Cmd  R General  Stf  Collage.  Ft  Leavenworth.  ATTN:  ATS*  oE-L 
1 USA  Cmd  R General  Stf  College,  Ft  Leavenworth,  ATTN:  Ed  Advisor 
f USA  Combined  Arms  Cmbt  Oav  Act.  Ft  Leavenworth,  ATTN:  OepCdr 
1 USA  Combined  Arm»  Cmbt  Das  Act.  Ft  Leavenworth.  ATTN:  CCS 
1 USA  Combined  Arms  Cmbt  Oav  Act,  Ft  Leavenworth.  ATTN:  ATCASA 
I USA  Combined  Armi  Cmbt  Oav  Act,  Ft  Leavenworth,  ATTN:  ATCACO-E 
1 USA  Combined  Arms  Cmbt  Dev  Act.  Ft  Leavenworth,  ATTN:  ATCACC-CI 
1 USAECOM.  Night  Vision  Lab.  Ft  Belvolr.  ATTN:  AMSEL-NV-SD 

3 USA  Computer  Sys  Cmd.  Ft  Belvoir,  ATTN:  Tach  Library 
1 USAMEROC,  Ft  Belvoir,  ATTN:  STSFB-00 

I USA  Eng  Sch.  Ft  Belvoir,  ATTN:  Library 
1 USA  Topographic  Lab,  Ft  Belvoir.  ATTN:  ETL  TD-S 
1 USA  Topographic  Lab,  Ft  Belvoir,  ATTN:  STINFO  Canter 
1 USA  Topographic  Lab.  Ft  Belvoir.  ATTN:  ETL-GSL 
1 USA  Intalliganca  Ctr  R Sch,  Ft  Huechuca,  ATTN:  CTD-MS 
1 USA  Intalliganca  Ctr  R Sch.  Ft  Huechuca.  ATTN:  ATS-CTD  -MS 
1 USA  Intalliganca  Ctr  R Sch,  Ft  Huechuca,  ATTN:  ATSI-TE 
1 USA  Intalliganca  Ctr  R Sch.  Ft  Huachuca.  ATTN:  ATSI-TEX--GS 
1 USA  Intalliganca  Ctr  R Sch,  Ft  Huachuca.  ATTN:  ATSI-CTS-OR 
1 USA  Intelligence  Ctr  R Sch,  Ft  Huachuca.  ATTN:  ATSI-CTD-DT 
1 USA  Intalliganca  Ctr  R Sch,  Ft  Huachuca.  ATTN:  ATSI-CTD-CS 
1 USA  Intalliganca  Ctr  R Sch,  Ft  Huachuca,  ATTN:  OAS/SRO 
f USA  Intalliganca  Ctr  R Sch.  Ft  Huachuca.  ATTN:  ATSI-TEM 
t USA  Intalliganca  Ctr  R Sch.  Ft  Huachuca.  ATTN:  Library 

1 COR,  HQ  Ft  Huachuca.  ATTN:  Tach  Ref  Olv 

2 COR,  USA  Electronic PrvgGrd,  ATTN:  STEEP-MT-S 
1 COR,  Project  MASSTER,  ATTN:  Tech  Info  Canter 

t Hq  MASSTER.  USATRADOC.  LNO 
1 Raaaarch  Institute,  HQ  MASSTER,  Ft  Hood 
1 USA  Recruiting  Cmd.  Ft  Sherdian,  ATTN:  USARCPM-P 
1 Senior  Army  Ads..  USAF  AGOOTAC,  Elgin  AF  Aux  Fid  No.  9 
1 HQ  USARPAC,  DCSPER.  APO  SF  96668.  ATTN:  GPPE-SE 
t Stimion  Lib.  Academy  of  Health  Sclencei,  Ft  Sam  Houston 
1 Marina  Corps  Inst.,  ATTN:  Dean-MCI 
1 HOUSMC.  Commandant.  ATTN:  Code  MTMT  SI 

1 HOUSMC,  Commandant,  ATTN:  Coda  MPI-20 

2 USCG  Academy,  New  London,  ATTN;  Admission 
2 USCG  Academy,  New  London.  ATTN:  Library 

1 USCG  Training  Ctr,  NV.  ATTN:  CO 
I USCG  Training  Ctr,  NV.  ATTN : Educ  Svc  Ofc 
1 USCG,  Psychol  Rat  Br.  DC.  ATTN:  GP  1/B2 
1 HO  Mld-Rangr  Br  MC  Oat.  Ouentico,  ATTN:  PRS  Olv 
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1  US  Mann*  Coipi  Liurton  01c,  AMC  Alexandria,  ATTN:  AMCGS  F 
1 USATRAOOC,  ft  Monro*.  ATTN:  ATRO  -ED 
6 USATRAOOC.  Ft  Monro*.  ATTN  ATPR  AD 
1 USATRAOOC.  Ft  Monro*.  ATTN  ATTS-EA 

1 USA  Fotc*i  Cmd.  Ft  MePherton.  ATTN:  Library 

2 USA  Aviation  Tett  Bit.  Ft  Rucker.  ATTN:  STEBGPO 

1  USA  Agey  tor  Aviation  Sataty.  Ft  Ruckar.  ATTN'  Library 

1 USA  Agey  tor  Aviation  Sataty,  Ft  Ruckar,  ATTN:  Educ  Adviaor 
t USA  A natron  Sch  Ft  Ruckar,  ATTN:  PO  Drawer  0 

t HQUSA  Aviation  Svi  Cmd.  St  Louii,  ATTN:  AMSAV-ZOR 

2 USA  Aviation  Syr  T*«t  Act..  Edward*  AFB,  ATTN:  SAVTE-T 
1 USA  Air  Oat  Scb.  Ft  Bliu.  ATTN:  AT5A  TEM 

1 USA  Air  Mobility  Rich  A Oav  Lab.  Moft«tt  Fid.  ATTN:  SAVDL  AS 
1 USA  Aviation  Sch.  Rat  Tnq  Mqt.  Ft  Ruckar.  ATTN:  ATST-T-RTM 
t USA  Aviation  Sch,  CO.  Ft  Ruckar.  ATTN:  ATST-O-A 
1 HO.  USAMC,  Alexandria.  ATTN.  AMXCD-TL 
t HO.  USAMC.  Alexandria.  ATTN:  COR 
1 US  Military  Academy,  Watt  Point,  ATTN:  Serial*  Unit 
1 US  Military  Acadamy.  Wart  Point.  ATTN:  Otc  ot  Milt  Ldrthp 
1 US  Military  Academy,  Watt  Point.  ATTN:  MAOR 
1 USA  Standardisation  Gp.  UK.  FPO  NV,  ATTN:  MASE-GC 
1 Otc  ot  Naval  Rich,  Arlington.  ATTN:  Coda  462 

3 Otc  ot  Naval  Rich.  Arlington.  ATTN:  Coda  468 
I Otc  ot  Naval  Rich,  Arlington.  ATTN:  Cod*  460 
t Otc  ot  Nava I Rich.  Arlington.  ATTN:  Coda  441 

1 Naval  Aarotpc  Mad  Ra*  Lab.  Paniacola.  ATTN:  A com  Sch  Oiv 
1 Naval  Aarotpc  Med  Rat  Lab.  Paniacola.  ATTN:  Coda  L61 
1 Naval  Aarotpc  Med  Ret  Lab.  Paniacola,  ATTN:  Coda  L6 
1 Chiat  ot  NavPart.  ATTN.  Pan-OR 
1 NAVAIRSTA.  Norfolk.  ATTN:  Sataty  Ctr 
t Nav  Oceanographic,  DC.  ATTN:  Coda  6261,  Chart*  ft  Tech 
t Can  tar  ot  Naval  Anal,  ATTN:  Doc  Ctr 
1 NavAirSytCom.  ATTN:  AIR-631X 
1 Nav  BuMad.  ATTN:  713 
1 NavHalicoptarSubSqua  2,  FPO  SF  96601 
1 AFHRUFTI  William  AFB 
1 AFHRL  ITT)  Lowry  AFB 

1 AFHRL  IASI  WPAFB.  OH 

2 AFHRL  (OOJZ)  Brook*  AFB 

1 AFHRL  IDOJNI  Lackland  AFB 
1 HCKJSAF  (INVSOI 
1 HOUSAF  (OPXXAI 

1 AFVTG  (R01  Randolph  AFB 

3 AMRL  IHEI  WPAFB.  OH 

2 AF  Intt  ot  Tech,  WPAFB.  OH.  ATTN:  ENE/SL 
1 ATC  IXPTDI  Randolph  AFB 

1 USAF  AaroMad  Lib.  Brookt  AFB  (SUL-4),  ATTN.  DOC  SEC 
t AFOSR  (NL).  Arlington 

1 AF  Log  Cmd.  McClellan  AFB,  ATTN:  ALC/DPCRB 

1 Air  Fore*  Acadamy.  CO,  ATTN:  Dept  ot  Bal  Sen 
6 NavPart  ft  Oav  Ctr.  San  Otago 

2 Navy  Mad  Nauroptychfatric  Rich  Unit.  San  Diego 
1 Nav  Electronic  Lab.  Sen  Diego.  ATTN:  Ret  Lab 

1 Nav  TrngCan,  San  Oiego.  ATTN:  Cod*  9000-Lib 
1 NavPoitGreSch.  Monterey,  ATTN:  Cod*  66Aa 
t NavPoitGreSch,  Monterey,  ATTN:  Cod*  2124 
1 NavTrngEquipCtr.  Orlando.  ATTN:  Tech  Lib 
1 US  Oapt  ot  Labor,  DC.  ATTN:  Manpower  Admin 
1 US  Dept  ot  Juttica,  OC.  ATTN:  Drug  Entorc*  Admin 
1 Nat  Bur  ot  Standard*.  OC.  ATTN:  Computer  Into  Section 
1 Nat  Clearing  Houte  tor  MH  Info,  Rocked)* 

I Denver  Federal  Ctr.  Lakewood.  ATTN:  BLM 
12  Detenae  Documentation  Center 

4 Oir  Ptych.  Army  Hq,  Ruteell  Ofct,  Canberra 

1 Scientific  Advtr.  Mil  Bd,  Army  Hq,  Rutaell  Otc*.  Canberra 

1 Mil  and  Air  Attecb*.  Auitrian  Embaiay 

t Centre  da  Recherche  Dai  Ftcfurt.  Humaina  da  la  Da  tenia 
National*.  Brotealt 

2 Canadian  Joint  Staff  Wethington 

1 C/Air  Stall.  Royal  Canadian  AF.  ATTN:  Part  Std  Anal  Br 

3 Chiat,  Canadian  Oaf  Rich  Staff.  ATTN : C/CRDSWI 

4 Brltith  0*1  Staff.  Britlth  Emhaaty . Wathtngton 


1 Del  ft  Civil  Init  ot  Envrro  Median#,  Canada 
1 AIR  CRESS,  Kentington.  ATTN  Info  Syt  Br 
I Militaarpaykologitk  Tjanatt*.  Copahagan 
t Military  Attach*.  French  Emt.iuy.  ATTN  Dor-  Sec 
1 MedacinChat,  C.E.RP  A Arienel.  Toulon/Naval  franca 
1 Prln  Scientific  Oft.  AptU  Hum  Engr  Rich  Dr*.  Minntry 
ot  Detent*.  New  Delhi 

1 Pert  Rich  Otc  Library.  AKA.  (trael  Oefenaa  Force* 

1 Minittarit  van  Detenu*.  DOOPIKL  Afd  Sociaai 
Ptychologitcht  Zakan,  The  Hague.  Nether  lend* 


